Mood-Based Organization and Display of Instant Messenger Buddy Lists

ABSTRACT

A mood state may be modeled using mood information for a content selection (e.g., a digitally-encoded song) and/or by using mood information determined by how a user is interacting with a media player. For example, a playlist engine on a host may determine that a particular song is associated with an uplifting mood, thus determining that a user who has selected that particular song currently is in an uplifting mood, and correspondingly may select additional songs and advertisements consistent with, or responsive to, the uplifting mood. Mood information also may be used to present a mood state of users (e.g., co-users) in a network for display in a graphical user interface (GUI). For example, a user&#39;s screen name appearing in an America Online (AOL) Instant Messenger&#39;s Buddy List may indicate a determined user&#39;s mood, such as “happy”, “sad”, “silly”, or “angry.”

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part application of and claimspriority under 35 U.S.C. §119(e) to U.S. application Ser. No.11/025,881, filed on Dec. 30, 2004, entitled “Personalizing ContentBased on Mood” the entire contents of which are hereby incorporated byreference.

TECHNICAL FIELD

This document relates to the presentation of mood information.

BACKGROUND

Digital content is distributed on a wide variety of devices and in awide variety of formats. The digital content may include movies, music,slides, games and other forms of electronic content.

One way that users communicate over a distributed computer network isthrough an instant messenger service such as AOL Instant Messenger. Anynetwork program which allows communication between multiple networkusers may include a co-user display such as the buddy list interface forthe AOL Instant Messenger. A co-user display may include one or morerepresentations of co-users.

SUMMARY

In one general aspect, a graphical user interface is configured todisplay information about more than one co-user. The graphical userinterface includes one or more co-user elements. Each co-user elementincludes an identity element structured and arranged to enable a user toperceive a co-user identity and an online status of the co-useridentity. The graphical user interface also includes one or more moodelements structured and arranged to enable a user to perceive a moodassociated with the co-user identity.

With respect to at least the first general aspect, implementations mayinclude one or more of the following. For example, the identity unit maybe a screen name associated with the co-user identity or a nameassociated with the co-user identity.

The one or more mood elements may include a visual indicator related tothe mood associated with the co-user identity. The visual indicatorrelated to the mood associated with the co-user identity may include ananimation related to the mood associated with the co-user identity. Theanimation may be animated in response to a current mood associated withthe co-user identity or a change in mood associated with the co-useridentity. The visual indicator may include an icon displayed adjacent toa corresponding co-user element. The icon may visually reflect the moodassociated with the co-user identity indicated by the co-user element.The visual indicator may include an indicator of a mood category, suchthat a co-user element of at least one co-user identity associated withthe mood of the mood category may be displayed beneath the visualindicator.

The one or more mood elements may include a textual description of amood associated with the co-user identity. The one or more mood elementsmay be associated with the co-user element that includes an identityelement. The identity element may be formatted with text having a style,where the style may be a font, a color, and/or a size. The style may berelated to the mood associated with the co-user identity.

The mood associated with the co-user identity may be based on a moodvalue. The mood value may be represented as a multi-dimensional rating,such that each rating may be associated with a particular aspect of themood associated with the mood value. The graphical user interface may beconfigured such that the co-user elements are visually arranged based ona difference between the mood value associated with the co-user identityand a reference mood value.

The graphical user interface may include a user feedback elementstructured and arranged to receive user feedback regarding accuracy ofthe mood determined and associated with the co-user identity. The userfeedback element may be configured to reflect positive and negative userfeedback. The negative user feedback reflected by the user feedbackelement may be used to correct the mood determined and associated withthe co-user identity, while the positive and negative feedback reflectedby the user feedback element may be used to improve the accuracy of themood determining element.

The graphical user interface may include a privacy control that may bestructured and arranged to allow a co-user to selectively block thedisplay of the one or more mood elements from being perceived in thegraphical user interface. The graphical user interface may include acontact list for facilitating instant messaging communications. The moodassociated with a co-user identity may, include a mood of happy, sad,tired, worried, angry, mad, busy, nervous, anxious, content, depressed,lonely, sick, annoyed, frustrated, ecstatic, amused, silly and/or lazy.

In a second general aspect, co-users having a particular mood aredetermined based on a search among a group of co-users each having anassociated mood. A mood value for at least one co-user is accessed.Reference mood values and corresponding moods are accessed. The moodvalue for the co-user is compared with the reference mood values. Basedon a result of the comparison between the mood value for the co-user andthe reference mood values a mood is associated with the co-user. Asearch query that includes at least some query parameter that is basedon mood information is received. The search query is applied to the moodassociated with one or more of the co-users. Which, if any, co-usershave an associated mood that satisfies the search query is determined.An indication of a co-user identity for co-users who have an associatedmood that satisfies the search query is returned.

With respect to at least the second general aspect, implementations mayinclude one or more of the following. For example, associating the moodwith the co-user may include, for at least one of several moods,determining whether the mood value for the co-user is within a range ofmood values associated with a mood with which reference values have beenaccessed. If so, the mood may be added to a list of moods associatedwith a co-user.

In a third general aspect, information about more than one co-user isdisplayed. One or more co-user elements is rendered. Each co-userelement includes an identity element structured and arranged to enable auser to perceive a co-user identity and an online status of the co-useridentity. One or more mood elements structured and arranged to enable auser to perceive a mood associated with the co-user identity isrendered.

With respect to at least the third general aspect, implementations mayinclude one or more of the following. For example, the mood associatedwith the co-user identity may be determined by calculating a mood valueassociated with the co-user identity, comparing the mood value to a listof moods and determining the mood associated with the co-user identitybased on the calculated mood value. The mood value associated with aco-user identity may be represented as a multi-dimensional rating, suchthat each rating may be associated with a particular aspect of the moodassociated with the mood value. The ratings may be associated with theparticular aspects of the mood and may be used as a basis to arrange theco-user elements. The mood values associated with the more than oneco-user identities may be used as a basis to arrange the co-userelements.

The more than one co-user elements may be arranged based on theircorresponding mood values to hierarchically present the co-userelements. As such, the user may perceive a difference in mood, asrepresented by the mood value, among more than one co-users associatedwith a same mood.

Implementations of any of the techniques described may include a methodor process, an apparatus or system, or computer software on acomputer-accessible medium. The details of particular implementationsare set forth below. Other features will be apparent from thedescription and drawings, and from the claims.

DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram of a communications system that enableselectronic content to be distributed.

FIG. 2A is a graph of a mood spectrum that illustrates how a selectionof content may be scored to quantify the mood in an automated manner.

FIG. 2B is a graph illustrating how a mood spectrum and scoring systemmay be used to associate content with an actual mood.

FIG. 2C is a graph illustrating a three-dimensional mood managementsystem that illustrates how mood consistency may be maintained usingthree or more factors.

FIG. 2D is a graph illustrating how mood transitions may incorporateintermediate tracks to create a more successful transition in reaching amood destination.

FIG. 3 is a block diagram of a mood-based playlisting system.

FIG. 4 is a flow chart showing how mood consistency may be maintainedbetween two tracks.

FIG. 5 is a flow chart showing how mood consistency may be maintainedusing a three-dimensional model to determine mood consistency.

FIG. 6 is a flow chart showing how a playlist of content may betransitioned from a mood originating point to a mood destination usingintermediate tracks.

FIG. 7 is an exemplary graphical user interface (GUI) that may bepresented to a user accessing a mood-based playlisting system.

FIGS. 8-10 illustrate exemplary GUIs that may be presented to a user intraining a mood recognition engine used in a mood-based playlistingsystem.

FIG. 11 is an exemplary listing of variables illustrating how facialcomponents may be used to generate a mood.

FIG. 12 is an exemplary user profile for a screen name that relates acomponent in a facial expression to a mood that may be used in amood-based playlisting system.

FIG. 13 illustrates an exemplary scripted sequence of operations for amood-based playlisting system configured to provide content usingmood-based selection criteria based in part on a camera to provide moodinformation.

FIG. 14 is a flow chart of an exemplary process by which a user's moodvariations may be recorded.

FIG. 15 is an exemplary GUI that illustrates how different inputs may beused as a mood sensor into a mood-based playlisting system.

FIGS. 16 and 17 are exemplary graphical user interfaces (GUIs)displaying a list of co-users organized by mood.

FIG. 18 is a visual representation of mood values in a three dimensionalspace.

FIG. 19 is an exemplary data structure for storing the mood values ofco-users.

FIG. 20 is an exemplary data structure configured to define moods interms of mood values.

FIG. 21 is an exemplary data structure for storing a list of co-users.

FIG. 22 is a flow chart showing an exemplary process by which a user maybe associated with a particular mood.

FIG. 23 is an exemplary graphical user interface (GUI) featuring co-userelements in a way representative of a mood.

FIG. 24 is an exemplary data structure of a configuration indicating howa mood may be expressed within a co-user element.

FIG. 25 is an exemplary graphical user interface (GUI) with co-userelements sorted in relation to a mood-based value.

FIG. 26 is an exemplary graphical user interface (GUI) enabling a userto configure how mood information may be shared.

FIG. 27 is an exemplary graphical user interface (GUI) configured toenable a user to provide feedback as to the accuracy of a calculatedmood value.

FIG. 28 is an exemplary graphical user interface (GUI) configured toenable a user to specify animations that may be presented with one ormore moods for a user.

DETAILED DESCRIPTION

A mood state may be modeled using mood information for a contentselection (e.g., a digitally-encoded song) and/or by using moodinformation determined by how a user is interacting with a media player.For example, a playlist engine on a host may determine that a particularsong is associated with an uplifting mood, thus determining that a userwho has selected that particular song currently is in an uplifting mood,and correspondingly may select additional songs and advertisementsconsistent with, or responsive to, the uplifting mood. Mood informationalso may be used to present a mood state of users (e.g., co-users) in anetwork for display in a graphical user interface (GUI). For example, auser's screen name appearing in an America Online (AOL) InstantMessenger's Buddy List may indicate a determined user's mood, such as“happy”, “sad”, “silly”, or “angry.” In another example, a user's moodmay be detected to be more subtly defined as a combination of moods or aparticular point on a spectrum of a mood from “happy” to “sad”, from“tired” to “energetic”, or from “good” to “evil.”

A user's mood may be determined based on express or implicit informationin a variety of ways. Mood information may be determined based oninformation expressly provided by the user. For example, a user mayprovide a current mood (e.g., “happy”) using a GUI, or by some otherform of input. Mood information also may be determined based oninformation expressly identified from content perceived as beingcurrently accessed by a user. For example, a user may be listening to aparticular song that is associated with an “unhappy” mood. Based on the“unhappy” mood of the content, the user's mood then may be determined asalso “unhappy.” Additionally, even if the content does not bear directlyon mood (e.g., a song without an associated mood), information relatedto the content may help determine a user's mood. For example, a mood maybe determined using metadata associated with content (e.g., a mood tagindicating “happy” that may be attached to the content), informationabout the content itself, such as text within a document or song lyrics(e.g., happy words, such as “smile,” “dance,” or “sunny”), informationrelated to the content file name or content title (e.g., “my happy song”as a content file name, or “love song” as a content title), attributesrelated to a source of the content (e.g., a radio station name orclassification, as well as the type of content being broadcast by theradio station (e.g., sports vs. music) if the content is a radiobroadcast), and/or general information (e.g., time of day or weather).

Once a user's mood is determined, the mood information may be used tocontrol playlisting. More particularly, a user's current mood may beenhanced or changed (e.g., transitioned) by careful selection ofcontent. For example, a user who is currently in a “content” mood may betransitioned to a “happy” mood by playing more up-beat songs. In anotherexample, a user who is currently in a “sad” mood may be transitioned toa “happy” mood by playing up-beat songs or songs from the user'sfavorite album or by the user's favorite artist. In both cases, playlistchanges may be subtle in order to easily transition the user's mood. Forexample, for a user currently in a “sad” mood, slightly more up-beatsongs may be played, followed by much more up-beat songs, followed bythe “cheeriest” music in the playlist to slowly transition the user to a“happy” mood.

In addition to using mood information to control playlisting, determinedmood information also may be displayed to other users (e.g., co-users)via a co-user list. A co-user list may have co-user elements and moodelements. Co-user elements relate to an identity of a co-user, such as ascreen name or nickname associated with the co-user. Mood elementsrelate to an indication of a current mood of a particular co-user. Moodelements may include an icon (e.g., a smiley face icon for a happyco-user and a frowning face icon for a sad co-user), a graphicalrepresentation of a mood in a co-user-centric interface (e.g.,presenting graphical mood information in response to a mouse-over orright-click on a co-user element), three-dimensional depiction of amood, an appearance of the co-user element (e.g., large, bright coloredlettering when a co-user is happy and small, dingy colored letteringwhen a co-user is sad), inclusion or exclusion from a group of co-users(e.g., all co-users in a happy mood are displayed in the co-user listunder a “happy” group), an indication of factors contributing to mood(e.g., it is raining at a co-user's location and hence the co-user is ina glum mood), explicit indication of one or more particular moods (e.g.,the word “happy” being placed next to the co-user element), and/or anidentification of a co-user's mood along one or more continuums of moodsor composite scores (e.g., an indication that a co-user is 90% happy orthat the co-user is happy, but not as happy as another co-user).

Furthermore, social networking aspects related to mood information alsomay be displayed in the co-user list. More particularly, co-users havingsimilar moods may be grouped together in the co-user list. For example,a co-user list may have different groups, such as “happy,” “sad,”“angry,” or “ecstatic” such that co-user elements associated withco-users having one of the mentioned moods may be located under theappropriate heading in the co-user list. In addition, within the moodgroups, a co-user's relative mood (e.g., mood relative to moods of otherco-users) may be displayed. For example, a co-user element may getbrighter as the co-user's mood intensifies, a co-user within the “happy”group who is the happiest of the co-users in the “happy” group may havebe listed at the top of the group, or a numerical indication (e.g., avector) of a co-user's mood may be displayed alongside the co-userelement to indicate the intensity of the co-user's mood.

Another social networking aspect related to mood information may includeblocking display or access to co-users on the co-user list based on moodinformation. For example, a user may set a preference for displaying andenabling communication exclusively with co-users in a happy mood. In oneconfiguration, only co-users in a happy mood appear on the co-user list.In another configuration, a user may set a preference for displayingonly co-users in the co-user list who are currently in the same mood asthe user. In this way, some, but less than all determined moods, may beoptionally displayed in a co-user list.

Furthermore, a user may set the display preferences on a co-user,co-user group, class of co-user, or entire community of co-users basis.For example, a user may choose to display all co-workers regardless oftheir mood, but may display friends, or a sub-set of all friends, basedon mood information. In another example, a user may set preferences on aper-co-user basis, such that a particularly annoying friend may only bedisplayed when that friend is in a good mood.

A user also may search for co-users having a particular mood. Forexample, a happy user may search for happy co-users. Users also may benotified when the mood of co-users changes. For example, a clip from ahappy song may be played when a co-user in the co-user list moves from asad mood to a happy mood. Additionally, a user may seek to influence themood of a co-user. For example, the user may tell a funny joke or send afunny picture in order to cheer up a co-user currently in a sad mood.

Identification of Mood and Playlisting Based on Mood

Digital content such as digitally encoded songs (e.g., MP3 and NSVfiles) and video may be accessed on a variety of platforms through avariety of distribution channels. Examples of the platforms includepersonal computers, specialized appliances (e.g., a compact digitalaudio player such as Apple's iPod™), home stereo systems, and otherdevices. Examples of the distribution channels include Internet radioand television stations, network-based on demand services, Internet andretail outlet purchasing, and promotional distribution (e.g., an opticaldisk provided in a magazine).

The plethora of digital content, distribution channels and contentproviders make it very easy for a user to identify and select differentcontent sources that are more responsive to the user's particularinterest at a particular time. For instance, to select contentresponsive to the user's particular interest at a particular time, auser's mood or emotional zone may be mathematically modeled. A mediaplayer (e.g., a jukebox or an application on a personal computer) may beused to select content responsive to the determined mood. The mood-basedplaylisting system may be used to select or plan a sequence of tracks(e.g., a playlist) to achieve or preserve a desired mood-state for theuser. A mood transition may be planned so that any mood change betweendifferent ‘tracks’ comports with a determined mood transitionspecification.

Content (e.g., a song or track) may be associated with a particular moodsuch that the content may be accurately selected in response to adetermined mood. In some implementations, a mood may be associated withparticular content based on objective criteria. That is, particularcontent may be associated with a mood independent of observed listeninghabits of a particular user. For example, a song entitled “Shiny HappyPeople” (by the band R.E.M.) may be associated with a happy mood becausethe word “happy” is in the song title and/or the melody is upbeat. Othercriteria may be used to associate particular content with a mood, suchas, for example, a genre related to the content (e.g., a blues song maybe associated with a sad mood) or individual song lyrics (e.g., thelyric “how sweet it is to be loved by you” may cause the James Taylorsong to be associated with a love struck mood). Alternatively, oradditionally, particular content may be associated with a mood based atleast in part on user attributes or listening habits. In thisconfiguration, for instance, a song may be associated with a particularmood because the song has been played by one or a threshold number ofusers identified as being in the particular mood while requesting orreceiving or listening to the song. For example, if a user in a happymood plays the song “Dancing on the Ceiling” by Lionel Richie, the songmay be associated with a happy mood and presented to other usersidentified as having a happy mood.

A likelihood exists that an actual mood for a user may differ from thepredicted mood for the user. For instance, a user may be listening toclassical music as the user is frantically packing last-minute for avacation. The output of classical music may indicate a relaxed mood forthe predicted mood, but the actual mood may differ, as the user isanxious, stressed, and/or panicked with last-minute vacationpreparations.

In order to increase the efficacy of a mood-based playlisting system, amood sensor such as a camera may be used to provide mood information tothe mood model. When the mood sensor includes a camera, a camera may beused to capture an image of the user. The image is analyzed to determinea mood for the user so that content may be selected responsive to themood of the user.

For example, a user listens to an interne radio station. The user mayinitially select one of several Internet radio stations before settlingon a 1980s-oriented station. The personal computer may include a desktopvideo camera that captures imagery of the user in front of the personalcomputer.

The 1980s-oriented station is flexibly configured to enable access todifferent content based on the mood of the user. Thus, if a user doesnot ‘like’ a first selection (e.g., WHAM's “Wake Me Up”), the user mayadvance to another selection, either explicitly (e.g., by selecting anext-track feature) or implicitly (e.g., by determining that the user isin a different mood). In particular, the media player or a host accessedby the media player may analyze imagery provided by the desktop videocamera and determine a mood for the user. When the user's facialexpression indicates disgust, anger, or upset moods, the media playermay select different content, for example, by selecting a differenttrack at the conclusion of the first selection, or by advancing toanother song altogether in the middle of the song.

In one implementation, the Internet radio station places anadvertisement responsive to a mood state, or controls the mood state ofprevious songs to place an advertisement. For example, no Internet radiostation may precede a desired advertisement with a specified sequence ofone or more content selections that foster the desired mood. A cameramay be used to evaluate whether the desired or predicted mood along thepreceding sequence represents an actual mood for the user. When theactual mood differs from the predicted mood, the media player may selecta different track or sequence of tracks to foster the desired mood orselects a different advertisement.

FIG. 1 illustrates a media-based communications system 100 that maydistribute content electronically. The media-based communications system100 includes a content source 110, a network 120, and a player 130.Although the media-based communications system 100 is shown as anetwork-based system, the media-based playlisting system may accessmedia files residing in a standalone device or in a differentconfiguration. For example, a mobile jukebox may play content in theform of music encoded in a media file format.

The content source 110 generally includes one or more devices configuredto distribute digital content. For example, as shown, the content source110 includes a server 112 and a duplicating switch 114.

Typically, a content source 110 includes a collection or library ofcontent for distribution. Alternatively, or in addition, the contentsource may convert a media source (e.g., a video or audio feed) into afirst feed of data units for transmission across the network 120. Thecontent source 110 may include a general-purpose computer having acentral processor unit (CPU), and memory/storage devices that store dataand various programs such as an operating system and one or moreapplication programs. Other examples of a content source 110 include aworkstation, a server 112, a special purpose device or component, abroadcast system, other equipment, or some combination thereof capableof responding to and executing instructions in a defined manner. Thecontent source 110 also may include an input/output (I/O) device (e.g.,video and audio input and conversion capability), and peripheralequipment such as a communications card or device (e.g., a modem or anetwork adapter) for exchanging data with the network 120.

The content source 110 includes playlisting software configured tomanage the distribution of content. The playlisting software organizesor enables access to content by a user community. For example, thecontent source 110 may be operated by an Internet radio station that issupporting a user community by streaming an audio signal, and mayarrange a sequence of songs accessed by the user community.

The playlisting software includes mood-based playlisting software thatmaintains a consistent mood in selecting content. Generally, themood-based playlisting software selects content so that any related moodtransition between different content components is acceptable.

The content source includes a duplicating switch 114. Generally, aduplicating switch 114 includes a device that performs networkoperations and functions in hardware (e.g., in a chip or part of chip).In some implementations, the duplicating switch may include an ASIC(“Application Specific Integrated Circuit”) implementing networkoperations logic directly on a chip (e.g., logical gates fabricated on asilicon wafer and then manufactured into a chip). For example, an ASICchip may perform filtering by receiving a packet, examining the IPaddress of the received packet, and filtering based on the IP address byimplementing a logical gate structure in silicon.

Implementations of the device included in the duplicating switch mayemploy a Field Programmable Gate Array (FPGA). A FPGA is generallydefined as including a chip or chips fabricated to allow a third partydesigner to implement a variety of logical designs on the chip. Forexample, a third party designer may load a FPGA with a design to replacethe received IP addresses with different IP addresses, or may load theFPGA with a design to segment and reassemble IP packets as they aremodified while being transmitted through different networks.

Implementations of the device included in the duplicating switch alsomay employ a network processor. A network processor is generally definedto include a chip or chips that allow software to specify which networkoperations will be performed. A network processor may perform a varietyof operations. One example of a network processor may include severalinterconnected RISC (“Reduced Instruction Set Computer”) processorsfabricated in a network processor chip. The network processor chip mayimplement software to change an IP address of an IP packet on some ofthe RISC processors. Other RISC processors in the network processor mayimplement software that monitors which terminals are receiving an IPstream.

Although various examples of network operations were defined withrespect to the different devices, each of the devices tends to beprogrammable and capable of performing the operations of the otherdevices. For example, the FPGA device is described as the device used toreplace IP addresses and segment and reassemble packets. However, anetwork processor and an ASIC are both generally capable of performingthe same operations.

The network 120 may include hardware and/or software capable of enablingdirect or indirect communications between the content source 110 and theplayer 130. As such, the network 120 may include a direct link betweenthe content source and the player, or it may include one or morenetworks or subnetworks between the content source and the player (notshown). Each network or subnetwork may include, for example, a wired orwireless data pathway capable of carrying and receiving data. Examplesof the delivery network include the Internet, the World Wide Web, a WAN(“Wide Area Network”), a LAN (“Local Area Network”), analog or digitalwired and wireless telephone networks, radio, television, cable,satellite, and/or any other delivery mechanism for carrying data.

The player 130 may include one or more devices capable of accessingcontent on the content source 110. The player 130 may include acontroller (not shown) that processes instructions received from orgenerated by a software application, a program, a piece of code, adevice, a computer, a computer system, or a combination thereof, whichindependently or collectively direct operations of the player 130. Theinstructions may be embodied permanently or temporarily in any type ofmachine, component, equipment, storage medium, or propagated signal thatis capable of being delivered to the player 130 or that may reside withthe controller at player 130. Player 130 may include a general-purposecomputer (e.g., a personal computer (PC) 132) capable of responding toand executing instructions in a defined manner, a workstation, anotebook computer, a PDA (“Personal Digital Assistant”) 134, a wirelessphone 136, a component, other equipment, or some combination of theseitems that is capable of responding to and executing instructions.

In one implementation, the player 130 includes one or more informationretrieval software applications (e.g., a browser, a mail application, aninstant messaging client, an Internet service provider client, or an AOLTV or other integrated client) capable of receiving one or more dataunits. The information retrieval applications may run on ageneral-purpose operating system and a hardware platform that includes ageneral-purpose processor and specialized hardware for graphics,communications and/or other capabilities. In another implementation,player 130 may include a wireless telephone running a micro-browserapplication on a reduced operating system with general purpose andspecialized hardware capable of operating in mobile environments.

The player 130 may include one or more media applications. For example,the player 130 may include a software application that enables theplayer 130 to receive and display an audio or video data stream. Themedia applications may include controls that enable a user to configurethe user's media environment. For example, if the media application isreceiving an Internet radio station, the media application may includecontrols that enable the user to select an Internet radio station, forexample, through the use of “preset” icons indicating the station genre(e.g., country) or a favorite. In another example, the controls mayenable the user to rewind or fast-forward a received media stream. Forexample, if a user does not care for a track on a particular station,the user may interface with a “next track” control that will queue upanother track (e.g., another song).

The media application includes mood-based playlisting software. Themood-based playlisting software may work independently of, or inconjunction with, playlisting software residing on the content source110. The mood-based playlisting software may mitigate the moodtransition created when the content changes. In one example, theplaylisting software permits the user to select from a recommended listof content that is consistent with the previous or present track. Inanother example, the mood-based playlist software may seamlessly managethe transition of content.

FIGS. 2A-2D describe a mood modeling system that may be used by thesystems described with respect to FIG. 1. FIG. 2A illustrates a moodspectrum 200 that may be used to determine a mood consistency between aselection of content and planned future content. Mood spectrum 200 hasbeen abstracted to be independent of the underlying mood, and has beennormalized in the range from 0 to 10. In mood spectrum 200, the moodindicator 205 for the current track has a value of approximately 5 onthe mood spectrum 200. The mood indicator 205 for the current track isrelated to the mood spectrum 210 consistent with the current track,which indicates mood values for content that may be selected consistentwith the mood value for the current track under consideration. In oneexample, the playlist and content selection is being planned and thecurrent track under consideration has not been distributed. In anotherexample, the current track under consideration has been or is beingdistributed (e.g., across the Internet by an Internet radio station).

FIG. 2B illustrates a graph 220 how content may be categorized using oneor more moods and specifically describes how the mood indicatorassociated with a particular piece of content may span multiple moods.As shown, the moods include “angry,” “excitement,” “dance,” “romantic,”“mellow,” and “sad.” FIG. 2B uses a 1-dimensional axis to categorizecontent along the mood spectrum 225. Specifically, the content in FIG.2B spans two of the moods, specifically, dance and romance. Otherdimensioning systems relating to more than two moods may be used. Forexample, an X dimensional system may gauge X moods across X axes.Nevertheless, regardless of the number of axes that are used, aselection of content may be related to various moods to identify futurecontent that is consistent with the mood of the content that has beenselected.

FIG. 2B includes a mood indicator 230 for the current track. The moodindicator 230 describes a particular mood rating for a piece of contentthat has been identified. The content that has been identified mayinclude a selection of content that is actually being played or one thatis destined for one or more users. Alternatively, the mood indicator fora current track may be used to create a user playlist to better identifydesired content deemed compatible for a user. As is shown in FIG. 2B,the mood indicator 230 for the current track lies within the moodspectrum 225 consistent with the current track. This mood spectrum 225indicates that content that falls within dance and romantic themes isdeemed consistent with the mood indicator for the current track.

In one implementation, the consistency with the current track and theidentification of a particular mood spectrum may be determined byscoring the current track and a proposed next track and determining therelationship between the score for the current track and the score forthe proposed next track. Alternatively, a selection of content may beassociated with one or more discrete values that describe the content.For example, a song may be associated with letters, each of whichdescribes one or more themes that may be used to characterize the song.Thus, as is shown in FIG. 2B, if D and R were used to identify,respectively, dance and romantic themes, a record describing the currenttrack could have a D and an R in its record/metadata.

Referring to FIG. 2C, a three-dimensional mood management graph 240 isshown that illustrates how mood spectrum consistency may be determinedacross three factors, influences, or moods. Specifically, thethree-dimensional coordinate system for the current track 245 is shownwithin a three-dimensional volume describing the mood spectrum boundary250 as a function of three coordinates. Also shown is a first song 255that does not fall within the volume of the mood spectrum boundaries 250and a second song 260 that lies within the mood spectrum boundary 255.Thus, when content is being selected, if the mood spectrum boundary 250is being used as the determining criteria, song 255 may be excluded asit lies outside the mood spectrum boundary 250, while song 260 may beincluded in the playlist as it lies within the mood spectrum boundary250.

Depending on the implementation and the configuration, the mood spectrumboundary may represent a simpler function such as a cone or a sphere.For example, a sphere may be used to identify equidistant points thatfall within a certain mood range of the current track. However, the moodspectrum boundary 250 need not include a simple function. For example,if detailed analytics are used to measure mood spectrum consistency anduser response, a more detailed and non-symmetrical volume may be used tomeasure the mood spectrum boundary 250. One illustration of this mayinclude content that may be very consistent across one axis for multiplethemes, but inconsistent with minor changes across a different axis inmood spectrum. For example, if the content is being scored acrosslyrics, tempo and intensity, lyrics that may contain age-appropriatesuggestions may only be consistent with content that is similarlyappropriate for the identified age. In contrast, content that features aslower tempo may be consistent with music across multiple themes with asimilar tempo. Accordingly, the function that describes the moodspectrum boundary 250 of the current track 240 may incorporate analyticsthat permit a small tolerable deviation in the lyrical deviation whilealso permitting a wider variation in the tempo axis.

FIG. 2D illustrates a graph of a three-dimensional mood consistencyscoring system 270 that illustrates how mood transitions may be plannedso that the mood may be changed from a current mood originating point toa mood destination. The transitions may be structured such that atransition directly from a mood originating point to a mood destinationthat otherwise appears difficult or unsuccessful may be made moresuccessful by using one or more intermediate transitions. Thus, thelikelihood of a successful transition between the mood originating pointand the mood destination point is increased.

Mood scoring system 270 illustrates a mood originating point 275 and amood destination 280. The general mood transition that is required isillustrated by the vector 285 from the mood originating point 275 to themood destination point 280. However, the mood consistency volume 277 formood originating point 275 does not include the mood destination point280. Accordingly, one or more intermediary tracks may be used tosuccessfully transition one or more users to the mood destination point.

To accomplish this transition, intermediary track 290 is used as thenext content selection to create a mood that is closer to the mooddestination point 280, even though the consistency volume 292 for theintermediary track 290 does not actually reach or include the mooddestination 280. After the intermediary track 290 is selected, a secondintermediary track 295 is added to the playlist to move the current moodindicator closer to the mood destination 280. As is shown in FIG. 2D,the intermediary tracks 290 and 295 both lie within the same transitionvolume 292, thus preserving a consistent mood transition from theintermediary track 290 to the intermediary track 295. From theintermediary track 295, the system may transition directly to the mooddestination point 280 and preserve the consistent mood as both theintermediary track 295 and the mood destination point 280 lie within themood transition volume 297.

Although the transition from the mood originating point 275 to the mooddestination point 280 features the use of two intermediary tracks, theimplementation of a successful transition need not be limited to the twointermediary tracks that are shown. For example, depending on theconfiguration, no intermediary tracks may be required to successfullytransition from the mood originating point 275 to the mood destinationpoint 280. Alternatively, one, two, three, or more intermediary tracksmay be used to successfully transition from the mood originating point275 to the mood destination point 280.

The intermediary tracks need not resemble similar forms of content. Forexample, the mood originating point for the current track may include asong that is being transmitted, a first intermediary track may include acommercial, a second intermediary track may include a second song, andthe mood destination point may relate to a planned advertisement thathas been targeted for increased chances of success.

Also, the volumes that describe the mood consistency may be configuredto reflect probabilistic chances of success and may change, based on thedesired chance of success. For example, the mood consistency volume 277may be planned on a model of mood consistency such that transitioningfrom the mood originating point 275 to the intermediary track 290 willpreserve 90% of the audience upon that transition. Alternatively, iffewer intermediary tracks are desired, a larger mood consistency volumethat covers more distance may be used based upon a modeled probabilityof 50%. Thus, in this model, fewer intermediary tracks may be requiredto reach the mood destination point.

Finally, the transitions that are shown may include real-time feedbackto better predict the actual user response to be transitioned. Forexample, a test audience may be sent the intermediary track in advanceof the larger audience. If the response of the test audience indicatesthat the transition is not as successful as was expected, an alternatepath may be plotted to increase the chance that the transition willpreserve the audience. For example, an intermediary track may be chosenthat lies closer to the mood originating point. Another example of analternative path that may be chosen includes a trusted transition thathas been used previously and is associated with what is believed to be ahigher success rate in transitioning.

FIG. 3 illustrates a mood-based playlisting system 300 that may be usedto generate a playlist with consistent moods between two or moreselections. The mood-based playlisting system 300 includes acommunications interface 310, a playlist manager 320, a content library330, a mood indicator library 340, a mood calculator 350, and anoptional mood-modeling engine 360. Generally, the mood base playlistingsystem 300 manages the playlist for one or more pieces of content to betransmitted to an audience. The communications interface 310 receivesdata describing the audience and one or more content goals to beincorporated, so that the playlist manager 320 may put together aplaylist of selections from the content library 330 by using the moodindicator library 340 to determine a score for the content andmaintaining consistency between the selected content using the moodcalculator 350.

The communications interface 310 may be used to exchange data describingthe audience that is being managed and/or to distribute playlistinformation. The communication interface 310 also may be used to receiveupdates to the content library 330, the mood indicator library 340, anddifferent algorithms and models used by the mood calculator 350.

The communications interface 310 receives updates from one or morepartners or other devices to exchange content for incorporation into aplaylist. For example, a newly-released song may be distributed, alongwith advertisements for incorporation into the playlist. Similarly, moodindicator information related to the content and/or advertising to bedistributed also may be received by the communications interface 310 fortransmission to the mood indicator library 340. Audience data associatedwith content may be modeled, described electronically, and transmittedto the mood calculator 350 to better select content to be incorporatedinto the playlist. The playlist manager 320 includes a code segment thatidentifies content to be used in a playlist. For example, the playlistmanager 320 may generate a playlist that describes a piece of content tobe accessed and reference information so that the content may beaccessed using the reference information.

Alternatively, the playlist manager 320 may generate a playlist to beused by a distribution point. For example, an Internet-based radiosystem may receive the playlist from the playlist manager fortransmission to the listening audience. Depending on the configurationof the mood-based playlisting system and whether the mood-basedplaylisting system is determining the playlist and distributing thecontent, the playlist manager 320 also may transmit the content to beused in the playlist (e.g., through communications interface 310).

The content library 330 may include one or more selections of contentfor incorporation into a transmission for a receiving audience.Depending on the nature of the content, the content library may beadjusted to accommodate the particular media and/or audio demands. Forexample, the content library may include digitally encoded songs andrelated music videos for broadband users. The content library also mayinclude metadata that describes the content. In the case of songs, themetadata may include, for example, artist, album, and track information.When the content library includes video information, the videoinformation may include different bit rates for different audiences.Thus, a user with a high bandwidth connection may be able to access aselection encoded for a higher bit rate and having relatively higherquality, while a user with a slower connection may be able to access thesame content encoded using a lower bit rate and having relatively lowerquality. The content library and the metadata in the content libraryalso may be associated with one or more rules that may be used in thecontent selection. Thus, a particular selection of content in thecontent library may have detailed licensing information that governs howthe selection of content may be accessed. For example, a particularselection of content may be available for promotional purposes during alimited time and may be unavailable thereafter. Other examples ofrestrictions that may be incorporated in the content library includeASCAP licensing restrictions that control the number of times aselection or content may be accessed in a particular period, andpreclude a selection of content from being accessed in a particularmanner. For example, a selection of content may be precluded from beingincorporated in a playlist twice in a row.

The mood indicator library 340 may include one or more values designedto describe the mood for a selection of content. Depending on theconfiguration of the mood-based playlisting system, different metricsmay be stored in the mood indicator library 340. Thus, one example ofthe value stored in the mood indicator library may describe a selectionof content and a mood indicator that scores the content in a specifiednumerical range. Another metric may include different values thatindicate whether a selection of content is compatible with a chosentheme or genre.

Although the mood-based playlisting system has been described asmaintaining consistency within a desired mood for a selection ofcontent, other non-mood-based elements may be modeled and incorporatedinto the content selection process and stored in the mood indicatorlibrary. For example, a user pool may be divided into premium andnon-premium communities. The premium community may be allowed to accessexclusive content that is not available to the non-premium community.This premium status for content that may be available may be stored inthe mood indicator library. Other non-mood-based metrics may be used.

For example, the mood indicator library 340 may include advertisingeffectiveness data. Examples advertising effectiveness data may include,but are not limited to, an indication of which advertisements should beused with specified moods, the effect of using an advertisement with thedifferent moods, projected and past advertisement efficacy (bothretaining a user and receiving a response) for both a user and ademographic, and reimbursement.

The mood indicator library 340 may be configured to incorporate feedbackbased on a user or a community of user's response to content. Forexample, the actual response by users to content may be tracked so thatefficacy data may be updated to reflect the users' actual responses.While an original data set may be used to predict a mood, the users'prior actions may be used in generating and consulting a mood model thatmore accurately predicts the users' response. Old rules that are notaccurate may be replaced by new rules determined to be more accurate.The new rules may be used to generate future playlists and/or selectcontent in the future for the user.

The mood calculator 350 may be used to receive values describing acurrent playlist, access the mood indicator library 340, and assist theplaylist manager 320 in generating the playlist. Depending on theconfiguration of the playlist manager 320, the structure of the moodcalculator 350 may differ. For example, in one configuration, theplaylist manager 320 may suggest a particular piece of content and pollthe mood calculator 350 to determine if the selection of content isappropriate and consistent with the current mood. The mood calculatorthen may respond with an indicator of whether the suggested content isconsistent.

Alternatively, the playlist manager 320 may provide an indicator of acurrent track that is being transmitted and may poll the mood calculator350 for a suggested piece of content. In response, the mood calculator350 may poll the mood indicator library 340 to retrieve a consistentpiece of content. The mood calculator 350 then may transmit the identityof the consistent content to the playlist manager 320, which mayretrieve the content from the content library 330.

As an optional element, the mood-based playlisting system 300 mayinclude a mood-modeling engine 360. For example, as content is beingadded to the content library 330, the mood-based playlisting system 300may interface with the mood-modeling engine 360 to determine and gaugethe mood spectrum for the newly-added content. The mood-modeling engine360 may use the communications interface 310 to develop an appropriatemood analytic for the newly added content. For example, the selectedcontent may be sent to a testing code segment to determine ananticipated user response. Alternatively, the mood-modeling engine 360may interface with the playlist manager to add the proposed content to atest group of listeners to gauge their response to the selected content.

Other analytics that may be used by the mood-modeling engine 360 mayinclude content analysis that may evaluate lyrics, the tempo, or otherelements relating to the content. In one example, the tempo for anewly-received piece of content may be “scored” using a frequencyanalyzer to determine the theme and mood with which the content isconsistent.

Although the mood-based playlisting system 300 is shown as aninterconnected group of sub-systems, the configuration of the mood-basedplaylisting system 300 may include elements that have allocated thefunctionality in a different manner. For example, the content library330 may be co-located or merged with the mood indicator library 340.Thus, the mood indicator for a selection of content may be stored as anelement of metadata residing with the content record. Alternatively, theelements described in mood-based playlisting system 300 may reside in alarger code segment with constituent code segments described by theelements shown in FIG. 3.

FIG. 4 is a flow chart 400 that illustrates how a track of content maybe selected in a way that maintains mood consistency. Specifically, theflow chart 400 may be implemented using the mood-based playlistingsystem such as was described previously. In general, a mood-basedplaylisting system determines a mood indicator that indicates a presentmood state of a user (step 410), determines a mood indicator describinga next track mood spectrum that is consistent with the mood indicatorfor the current track (step 420) and selects a next track that lieswithin the next track spectrum for the current track (step 430).

Initially, the mood-based playlisting system determines a mood indicatorthat indicates a present mood state of a user (step 410). Typically,this will include creating a score that describes the track of contentunder analysis. For example, a song being distributed on the radio couldbe given a score from 0 to 10. In a multi-dimensional scoring system,the mood indicator could include a multi-dimensional coordinate thatdescribes the mood indicator with regard to several variables.

The mood indicator may be determined in advance of distributing thetrack. For to example, the system may assemble a user playlist with asequence of tracks for distribution. This sequence may be distributed todistribution nodes (e.g., local radio stations or regional Internetservers). Alternatively, the mood indicator may be determined for atrack that is being or has been distributed. For example, the moodindicator may be determined for a song that is being played over theairwaves.

A mood spectrum may be determined for the track for which a moodindicator has just been determined (step 420). The mood spectrum may beused to select the next track such that the next track lies within theboundaries of the mood spectrum. As has been described previously, themood spectrum may include multiple variables and may relate to alikelihood of success that a user may stay with the current distribution(e.g., the same channel) upon the playing of the next content selection.

With the mood indicator and the mood spectrum for the current trackdetermined, a next track is selected that lies within the mood spectrum(step 430). In one implementation, the next track may be selected byfinding the track that is closest to the current track. For example, ifthe current track has a score of 5.17, the next closest track that maybe selected may have a score of 5.18.

Alternatively, a content programmer may wish to have some variationwithin a mood spectrum, and the selection criteria may include arequirement that the next song differ by more than a specified variationthreshold while still being within the specified mood spectrum. In theprevious example, the content could be selected to be at least 0.5 unitsaway from the current selection of 5.17 but still lies within thevariation describing the mood spectrum of 1.0.

Within the range of values that are acceptable, the content may beselected randomly or the content may be selected based on identifyingcontent that matches the criteria (e.g., is the furthest or closest awaywithin the spectrum). If there is not a track that lies within the moodspectrum, the mood-based playlisting system 300 may alter itsconfiguration to generate a selection of content. For example, the moodspectrum may be expanded so that more content lies within the moodspectrum. This may be accomplished by, for example, decreasing thethreshold percentage of a success that is required in the transition orincreasing the values that define the threshold for success. Forexample, if the mood spectrum only covered 70's rock, the mood spectrummay be expanded to include 70's and 80's rock.

FIG. 5 illustrates a flow chart 500 showing a mood-based playlistingsystem that incorporates a three-dimensional mood-based modeling system.In general, the three-dimensional mood-based playlisting system operatesby determining a coordinate mood location for a current track that isbeing played. This may include or be described as the present mood stateof a user. With the coordinate mood location determined, a compatiblemood volume may be determined that describes future content selectionsthat are deemed consistent with the present mood state. With thecompatible mood volume identified, one or more tracks that lie withinthe compatible mood volume may be identified and a user may be able toaccess the identified tracks.

Initially, a coordinate mood location that indicates the present moodstate of a content selection is determined (step 510). For example, themood state may be described on X, Y and Z axes. In one example, thecoordinate mood location is described in the context of the content thatis being distributed. For example, the mood coordinates may measure thesongs lyrics, tempo, and/or style. Alternatively, the coordinate moodlocation also may measure or describe the mood of the audience. Forexample, a particular song may be associated with a human emotion suchas sadness, joy, excitement, or happiness. The human emotions may bemeasured independent of the underlying theme of the music. For example,some music whose theme is described as “golden oldies” may be associatedwith sadness while other music may be associated with joy.

With the coordinate mood location determined, a compatible mood volumedescribing compatible and consistent future content may be determined(step 520). For example, a sphere around a coordinate mood location maybe identified that describes content compatible with the present track.With the compatible mood volume described, one or more tracks that liewithin the mood volume may be identified (step 530). With the trackidentified, a user may be enabled to access the identified track (step540).

In FIG. 6, flow chart 600 illustrates how an audience may betransitioned from an originating piece of content to a destination pieceof content. This may be used, for example, to transition a user from aparticular piece of programming (i.e., the originating content) to atargeted advertisement (i.e., the destination content) by tailoring thetransitions from the originating content to the destination content.Accordingly, the likelihood of success and the effectiveness of thetransition may be pursued.

Generally, the operations described in flow chart 600 may be performedusing the systems and models described with respect to FIGS. 1-3. Forexample, the mood-based playlisting system 300 may be used to generatethe playlist that transitions the user from the originating piece ofcontent to the destination. Similarly, the transition and intermediatetracks described in FIG. 2D may be used to increase the effectiveness ofthe transitions. However, depending on the characteristics of theoriginating and destination content, the selection of the mood-basedtransition path may differ.

Generally, a mood-based playlisting system identifies a mood destinationfor a user playlist. A mood originating point is determined. With theoriginating point and destination paths known, a mood transition may becalculated from the mood originating point to the mood destination.

Initially, a mood destination for a user playlist is identified (step610). Generally, identifying a mood destination includes identifying aselection of content to be included in the user playlist. For example, adistributor may wish to place a certain advertisement. Alternatively, asystem administrator may wish to have an optimal lead-in for aparticular piece of programming for the purpose of, for example,achieving optimal ratings for network content. This content to beinserted in the user playlist has an associated mood that relates to thecontent being distributed. In yet another example, a systemadministrator for a mood-based playlisting system may wish to have anoptimal lead-in to increase the effectiveness and response of theaudience to identified content that is to be transmitted in the future.

Separately or in parallel, a mood originating point may be determined(step 620). Determining a mood originating point may include identifyingcontent that is being distributed or will be distributed to an audienceprior to the transmission of the content associated with the mooddestination. A mood originating point may be determined for the contentthat is being distributed. If the mood originating point differs fromthe mood destination of the content being transmitted (or to betransmitted), the resulting differential may create a mood transitionthat may create a less responsive result due to differences in the moodsof the particular content. The mood transition from the mood originatingpoint to the mood destination is calculated (step 630). Depending on thevariation between the mood destination and the mood originating point,the transition may include one or more intermediary tracks. Theintermediary tracks may be selected no that the mood metric for theintermediary tracks lies within the mood-consistency spectrum or volumeof the previous track. Using the previous content or track as abaseline, the next content or track may be selected to minimize thenumber of intermediary tracks between the originating content and thedestination content.

FIG. 7 is an exemplary GUI 700 that may be presented to a user accessinga mood-based playlisting system. For convenience, particular componentsand messaging formats described earlier are referenced as performing theprocess. However, similar methodologies may be applied in otherimplementations where different components are used to define thestructure of the system, or where the functionality is distributeddifferently among the components shown.

GUI 700 includes track and media player controls 710, a mood indicator720, a mood preservation control 730, a mood correction control 740, andan advertisement display 750.

The track and media player controls 710 display the current and nexttracks, and enable the user to control a media player (e.g., volumecontrols, rewind or fast forward, advance to a next track). Track andmedia player controls 710 indicate that the current track is EltonJohn's Philadelphia Freedom and the next track is the “Theme fromRocky.”

The optional mood indicator 720 features a display that informs a useras to their mood state/status as determined by with respect to amood-based playlisting system. The mood indicator 720 greets the user byscreen name (“Hello SCREEN_NAME”), indicates the mood state (in thiscase that the user is determined to be depressed), and also indicatesthe mood destination (that the proposed playlist is designed to upliftthe user). The screen name may be used to personalize the mood-basedplaylisting system to a user's identity. For example, a furrowed browfor a first user may be associated with an upset mood while a similarlyfurrowed brow for a second user may be associated with a neutral or evenpleased mood. Although the analysis of moods or facial expressions maybe personalized, a mood-based playlisting system may use normativeanalysis that is believed to be valid for large portions of thepopulation. The normative analysis may be enhanced by analyzing multipleportions of a facial expression so that uncommon facialexpressions/moods may be accounted for.

The optional mood preservation control 730 enables a user to preserve apresent mood. In the example shown, the user may click on a “I want tostay depressed” button to select content consistent with the depressedmood.

The mood correction control 740 enables a user to specify that theiractual mood is different from a mood that has been attributed to theuser. For example, if a user is actually happy while listening to thecurrent track (“Philadelphia Freedom”), the user may access a drop downmenu (not shown) and indicate their actual mood, which in this case ishappy. As shown, the mood correction control 740 renders an image of theuser.

Activating the mood correction controls may enable a user to identify,select, or highlight one or more features in a facial expression. Forexample, in response to activating a drop down menu to indicate the useris happy, the media player may present the image (e.g., in moodconnection control 740) and ask the user to select one or more featuresassociated with a happy mood. The user then may interact with the imageand select a facial structure, such as a brow or smile, to indicate thatdetection of the selected facial structure reveals a particular mood.The indicated facial structure then may be stored and used in futureanalysis to identify a user's mood.

An optional image may appear in mood correction control 740 enabling auser to perceive a visual display. Presenting the visual display may beused in training the mood-based playlisting system to be responsive tothe actual mood of the user, or to illustrate to a user whether one ormore moods (e.g., a predicted mood) represents an actual mood.

The optional advertisement display 750 is used to present an image, oran audio/video clip. In the example shown, the optional advertisementdisplay 750 features a video advertisement for an energy drink withathletic images. In one implementation, the video clip is played whilethe audio content selection is playing. In another example, the videoclip is played upon the completion of the audio selection. Note that theadvertisement may be coupled to the mood, as indicated in GUI 700 wherean athletic advertisement is linked to an uplifting mood.

FIGS. 8-10 illustrate exemplary GUIs 800, 900, and 1000 that may bepresented to a user while in training a mood recognition engine used ina mood-based playlisting system. Although mood recognition engine may beable to adequately recognize a mood for a particular user, a trainingprogram may be used to make the mood recognition engine more accurate,account for a nonstandard facial expression/mood indication, and/or toidentify which components in a facial expression should be used todetermine a mood state. Generally, the exemplary training program, shownin GUIs 800, 900, and 1000, prompts a user to present a neutral, happy,and angry mood, as specified by mood prompter 810, 910, and 1010,respectively. Feedback displays 820, 920, and 1020 present an image ofthe user rendering the desired mood. In feedback display 820, a user'sneutral mood may be determined by detecting that (1) the brow structureand lips are parallel to the user's shoulder plane of the (parallel tothe horizon); (2) the lack of wrinkles around the mouth; (3) the neutralposition of the eyelid; and (4) the lack of tension present in thecheeks. In contrast, feedback display 920 illustrates how the presenceof the user's happy mood may be determined by detecting (1) the presenceof teeth, elliptical structure of the mouth, and pronounced angularstructure between the mouth and the nose indicate a smile, and thus, ahappy mood state; (2) the eyelids are retracted; and (3) the browstructure is curved around the eye. Finally, feedback display 1020illustrates how the presence of the user's angry mood may be determinedby detecting (1) the declining dip at the peripheral of the mouth; (2)the tension of muscles in the forehead and the cheek; (3) theorientation of the eyelid over the eye; and (4) the downward structureof the interior portion of the brow over the nose. Note that FIGS. 8-10illustrate how both frontal and portrait (from the side) images may beused to determine the actual mood.

The pronouncement of the features used to identify a mood may varybetween users. For example, in some users, anger may not be determinedunless pronounced wrinkles are detected in the forehead, while in otherusers (e.g., the user shown in FIG. 10) minimal tension may be used todetermine anger. Also, FIGS. 9 and 10 illustrate that even when a commonfeature is present in more than one mood (e.g., the presence of thepronounced angular structure between the peripheral of the mouth and thenose), other facial features may be used to infer mood. In response toviewing the feedback display, a user may alter a facial expression topresent a mood likely to be responsive and/or likely to resemble anactual mood.

The user may provide additional mood determination information usingalternate sensor gateways 830, 930, and 1030. The alternate sensorgateway allows a user to enter a heart rate, voice print and/or brainwave during the mood training process so that the heart rate, voiceprint, and brain wave metrics may be used in the future to betterdetermine a mood.

Although FIGS. 8-10 described a user actively training a moodrecognition engine, a mood-based playlisting system may use passivetraining techniques and/or more subtle training techniques. Toillustrate a passive training system, a mood-based playlisting systemmay play a content selection, and analyze the responsive facialexpressions as a baseline indicative of a particular mood. In a slightlymore active training system, a content selection is played withoutasking the user to present a particular expression during which an imageof the user is captured. In response to the content selection, and/orperceiving their image, the user is prompted to indicate a mood duringthe content selection. The mood-based playlisting system then may askthe user if the image is indicative of the mood provided by the user.Moreover, if the image is not indicative of expressed mood, the user mayadvance through a series of images captured during the rendering of thecontent selection to identify an image associated with the indicatedmood. Another example used separately or addition to previouslydescribed examples allows a user to identify one or more components in afacial expression indicative of the desired mood (e.g., by allowing theuser to highlight a brow structure, a lip structure such as a smile, oran existence of wrinkles in a particular portion).

FIG. 11 is an exemplary listing of variables 1100 illustrating howfacial components may be used to generate a mood. Although listing 1100relates to an exemplary configuration variables, the listing 1100 alsomay be presented in a GUI enabling a user to identify which componentsmay be used in a mood-based playlisting system.

Listing 1100 includes hair-related descriptors including a position ofhairline 1110, a presence of hand on the hair 1111, and a presence/typeof hat 1112. Examples of using the position of the hairline 1110 mayindicate an incredulous/surprised mood when the position of the hairlineis forward, a stressed/expressed expression when the hairline is pulledback, and an indeterminate mood when the hairline is in a neutralposition. The presence of a hand on the head 1111 may indicate astressed mood (e.g., pulling hair out in frustration), tired (e.g.,running two hands through the hair), or ‘cool’ (e.g., running one handthrough the hair). A presence/type of hat indicator 1112 may indicate anathletic mood when the hat is a baseball cap (e.g., upbeat, excited),cockiness/arrogance (e.g., wearing a hat backwards), or formality (e.g.,a Sunday bonnet).

The presence/existence/position of wrinkles 1120 may indicate moodand/or state, for example, through the indication of pain when wrinklesappear in the cheek, happiness when angled wrinkles appear around asmile, skepticism/bewilderment when wrinkles appear in the forehead, andunease when concentric wrinkles appear around the mouth.

The presence/position of facial muscles and/or of tension in the muscles1130 may be used to indicate intensity or determination when the lateralfacial or forehead muscles are tense, or relaxedness/contentment whenthe muscles are not being used.

The orientation of the eye structure 1131 may indicate unease/skepticismwith a squint, shock or unbelief with an open eye structure, anger witha slight squint, and a neutral or content mood with a normal eyeorientation. Eye structure may be determined by identifying the relativepositioning between constituent components (e.g., eyelash, eye line,eye-originating wrinkles, and/or the eye itself).

The brow 1140 may indicate anger/skepticism/dislike when furrowed,surprise when raised, and neutrality/happiness when raised. The angle ofa furrow may indicate an intensity and/or distinguish betweenanger/skepticism/dislike.

The mouth structure 1150 may indicate whether a user is responsive to acontent selection by mouthing the words, such as the case when thefacial structure changes at a frequency appearing in a contentselection. The angle of the lips and side of the mouth 1151 and thepresence of teeth 1152 may further refine/identify a mood. A smile mayindicate happiness, a frown may indicate unhappiness, yelling mayindicate anger, and clenched lips may indicate intensity or discomfort.

The presence and/or orientation of facial hair may be used to indicate amood, or used in conjunction with other components described previously.For example, a presence of muscles may be difficult to determine due tominimal variation in skin color or surface location. However, thetracking the movement of facial hair that mimics or corresponds to theunderlying component may be easier to detect given the texture inherentin many forms of facial hair.

In addition to using a component (e.g., a beard, a mustache, a style ofglasses, abut, an earring, a piercing, and/or a tobacco product) toidentify a mood, the presence or absence of a component may be used toidentify a personality. For example, a goatee may be used to indicate aneasygoing personality with a preference for jazz, a full beard may beused to indicate a preference for country, sunglasses may be used toindicate a personality striving for a cool appearance, a set of bifocalsmay be used to indicate a preference for older or classical genres ofmusic, an earring or piercing may be used to indicate a preference forcontent on the fringes, a pipe may be used to indicate a preference forclassical music, and a cigar may be used to indicate a preference fortalk radio.

FIG. 12 is an exemplary user profile for SCREEN_NAME that relates acomponent in a facial expression to a mood that may be used in amood-based playlisting system. The mood/facial components are describedin a programming construct that may appear in a configuration file orthat may be used as a script in a programming construct.

Rule 1210 indicates how a position of a hairline may be used todetermine a mood. In the example shown, a default rule indicates that aforward hairline indicates a relaxed mood. When the hairline is setback, a tense mood is inferred which includes the set of anger, anxiety,upset and upset moods. Rule 1211 indicates how the presence of a hand onhair includes a default rule where SCREEN_NAME is deemed tense when ahand lies on the hair. Similarly, rule 1212 indicates that when abaseball cap is being used, then a mood is believed to be relaxed orhappy.

Rule 1220 uses a presence and/or position of wrinkles to determine moodinformation. If wrinkles exist in the forehead, then the mood isdetermined to include a tense set that includes anger, anxiety, and/orupset moods. If vertical wrinkles exist outside a structure identifiedas a mouth, then the mood is determined to be anger. If verticalwrinkles exist outside of the eye, then the mood is determined toinclude pain.

Rule 1230 uses the presence and/or position of facial features (e.g.,facial muscles) and/or of tension in the muscles to determine moodinformation. For example, if the cheeks are believed to be tense, thenthe mood may be determined to include business moods, which includes theset of anxiety and focus moods. In contrast, when the cheeks arebelieved to be relaxed, the mood may be described as NOT business andNOT tense moods (e.g., any or all moods except the moods found inbusiness and tense mood sets).

Rule 1231 indicates that the orientation of eye structure may be used todetermine a mood. In particular, if an eye is squinted, the mood may bedeemed non-responsive, which includes skeptical, dislike, and/or painmoods. If the eye is closed, the mood is determined to include slowmoods, which includes sleep, bored, and/or relaxed moods. If the eyesare believed to be wide opened, then the moods are determined to includewonder moods that include the moods of shock, amaze, and/or curiosity.

Rule 1240 indicates how brow information may be used in determining amood. For example, brow information may include a presence offurrowed/unfurrowed brow structures and/or relate to positioninformation. As shown, a furrowed brow is described as a downward browor a brow with an angle less than 135 degrees.

If a brow is furrowed, then the mood is determined to include a tensemood. In contrast, an unfurrowed brow is defined as an upward brow or abrow at an angle of less than 135 degrees. If the brow is unfurrowed,then the mood is determined to include a relaxed or happy mood.

Rule 1250 describes how the presence/position of a mouth structure maybe used to determine mood information. Rule 1251 indicates that theangle of lips and side of mouth may be used in determining a mood. Rule1252 indicates that the presence of teeth in an image can be used toidentify a mood. A smile is determined to exist when the end of themouth is pointed up or teeth are determined to be together. If a smileis identified, then a mood is determined to include relaxed and happymoods. However, if teeth are identified, but the teeth are not togetherand a microphone detects yelling, the mood is determined to includeangry moods.

As discussed earlier, rule 1260 indicates how the presence ororientation of hair facial hair may be used to determine a mood. Whendreadlocks exist, the mood is determined to be reggae. If the user isunshaven, a mood may not include classical or pop. If a full beard isdetected, then the mood may include rock, metal, or country music.

Rule 1270 indicates miscellaneous factors that may be used to indicate amood. For example, when a pipe is detected, the mood may includerelaxed, classical, jazz, and national public radio states.

Rule 1280 prioritizes between different moods. Depending on how theunderlying mood is modeled, elements of different moods may be detected.Prioritization provides rules that resolve competing, inconsistent, ordiffering mood states. For example, a business mood may be favored overa happy mood. A non-responsive mood may be favored over a tense mood. Ifchildren are detected (e.g., background audio signals are identified asscreaming or yelling) using the camera or microphone, then no metalmusic may be played.

Mood conflicts may be resolved using a number of different models. Inone model, a dominant mood is identified and content responsive to thedominant mood is selected in response. For instance, imagery provided bya camera may include a smile indicating happiness, strained facialmuscles indicating tension, and a set back hair line also indicatingtension. In one implementation of the dominant mood model, tension isidentified as a dominant mood since the majority of the detected moodsindicate the user is experiencing tension. In another variation, tensionmay be defined as a dominant mood over happiness by virtue a rule thatspecifies that tension should be used as the mood even when happiness isdetected. Identifying the dominant mood may include using a ranked listof moods, or a collection of relative mood preferences. Yet anothervariation may include deriving metrics for the magnitude of any one moodstate and comparing the metrics associated with the mood. Thus if a userhas a larger smile or maintains a smile over a longer duration whilemomentarily presenting a tense appearance, the happiness associated withthe smile of the longer duration may be identified as the dominant mood.

In another model, the mood-based playlisting system may select one ofseveral identified moods to accomplish the objective. For instance, ifthe mood-based playlisting system determines that a user may beexperiencing tiredness, happiness, and shock, the mood-based playlistingsystem may attempt to work with the happiness mood to realizeobjectives. When a user is non-responsive to content oriented towardsthe happiness mood, another mood may be used.

In one implementation, several models for resolving conflicts may beused. For example, a hybrid of models may be used so that, ifassimilating multiple models indicates a prevailing mood state, suggestsa particular transition, or identifies a particular content selection,the indicated state, transition, or selection may be used. Separately orin addition, if user responsiveness indicates that one model is moreeffective than another model, the more effective model may be used forthose configurations and environments for which the more effective modelis deemed effective. When configuration and environmental data indicatesthat another model is more effective, then the other model may be used.

Rule 1290 allows a mood state to be more customized, that is, moreprecisely tailored to associate content with a mood. The customizationmay be applied to a subscriber community, a listening audienceassociated with an Internet radio station, a demographic, or user. Forexample, rock has been modified to exclude ARTIST1 and ARTIST2. A talkmood has been modified to exclude TALK SHOW_HOST_(—)1. Happy includesMETAL and ROCK and does not include POP unless ARTIST3 is singing orSONG4 is provided.

FIG. 13 illustrates an exemplary scripted sequence of operations 1300for a mood-based playlisting system configured to provide content usingmood-based selection criteria based, relying in part, on a camera toprovide mood information. Generally, sequence 1300 represents theoperations that are performed and the results that are realized usingthe mood-based playlisting system with camera inputs. In oneimplementation, the sequence of operations 1300 represents actualentries appearing in a log used by a mood-based playlisting system. Inanother example, the sequence of operations 1300 represents a sequenceof procedural calls and resultant data.

Operation 1305 is a conditional function that specifies placement ofADVERTISEMENT1 in response to the condition that an emotional state, anintensity, a tempo, a genre lead to PLACEMENT_CRITERIA. For example,rule 1305 may be invoked in anticipation of or upon reaching of acommercial break at the end of a sequence of content selections.

Operation 1310 indicates that an advertisement for a sports drink shouldbe played when the mood is happy, intense, upbeat, and the song type(e.g., genre) is rock or hip hop. When the song type is rock, then arock advertisement should be placed. When the song type is hip hop, thena hip hop advertisement may be placed.

Operation 1315 provides a history for an online identity identified asSCREEN_NAME. The history may be used to understand or determine a user'sperceived mood state and/or recent transitions in mood. Thus, a user mayhave selected an 80's station, skipped the “The Lady in Red” and RunDMC, and listened to Van Halen in the preceding sequence. A preliminarymood determination is made using only the previous sequence. As aresult, operation 1320 predicts a mood state as anger, intense, upbeat,rock, not ballad (e.g., based on the user skipping the ballad “The Ladyin Red”), and not hip hop (e.g., based on the user skipping hip hopsongs by Run DMC). Thus, past perceived mood transitions may be used asa basis for future determination of how/whether/when to invoketransitions from their current mood to a desired mood. A log of pastperceived mood transitions may be recorded for a user (or community ofusers). The log may record a user's facial expressions captured duringthe course of the rendering content selections so that the perceivedmood transition is based on imagery-based mood state determinations.

Operation 1325 specifies that the mood should be confirmed using acamera. In operation 1330, the camera indicates that the actual mood isanger, neutral, non-responsive, and indeterminate.

Thus, as a result, the desired mood state may be related to the actualmood state. As shown in operation 1335, differences between the desiredmood state and the actual mood state may be used to indicate that theuser needs to transition from anger to happy, from neutral to intense,and from non-responsive to upbeat. If the genre is ok, and the existingmood may be used.

In operation 1340, the mood indicator library is queried to identify acontent selection that supports the transition.

In response to the query, in operation 1345, the mood indicator library(e.g., mood indicator library 340 in FIG. 3) returns data indicatingthat if a rock advertisement is placed, 50% of users will listen throughthe end of commercial. The mood indicator library also indicates thatadvertisers will not pay for a 50% retention. If Huey Lewis is used asan intermediary track before the commercial, then the retentionlikelihood becomes 75%. On the other hand, if the intermediary trackincludes Bon Jovi's “Living on a Prayer”, the likelihood is 90%. Thecost per Huey Lewis is $0.0001 per listener, the cost per Bon Jovi is$0.0011. The reimbursement from the advertiser is 75% while thereimbursement at 90% is $0.003

As a result, Bon Jovi living on a prayer is selected in operation 1355.The predicted mood is happy, intense, upbeat, and rock. The differencesin cost compared to efficiency need not be the determinative factor. Forexample, some advertisers may go to extraordinary lengths to preservebrand awareness or perception (e.g., is the advertised product deem“cool’). One measure of brand awareness may include retention rates.Thus, an advertiser concerned about brand perception may pay a premiumto ensure the highest retention.

In operation 1360, imagery data is analyzed to determine the actualmood. For example, data from the camera indicates that the actual moodis happy, intense, non-responsive, and rock. The predicted mood isrelated to the actual mood. The emotional state, tempo, and genre areacceptable. However, the intensity needs to change from non-responsiveto upbeat.

To change the intensity from non-responsive to upbeat, the moodindicator library is queried to provide a content selection thatsupports the mood transition. In operation 1370, the mood indicatorlibrary indicates that 85% will listen to entire commercial if the rockadvertisement is played now, and that 95% will listen to the entirecommercial if the Grateful Dead's “Touch of Gray” is played. As aresult, the advertiser reimbursement at 85% is $0.0025 while thereimbursement at 95% is $0.005. To realize the increased reimbursement,a “Touch of Gray” is played in operation 1375.

The camera is used confirm that the actual mood is in fact happy,intense, upbeat, and rock (in operation 1380), and that the rockadvertisement is played in operation 1385.

To confirm recipient response, the camera is used to generate imagerydata, which confirms that the actual mood during the rock advertisementwas happy, intense, upbeat, and rock. In operation 1390, the mediaplayer indicates that SCREEN_NAME listened to entire selection and thatthe user selected the advertisement, entitling advertiser to a bonus.

FIG. 14 is a flow chart of an exemplary process 1400 by which a user'smood variations may be recorded. Generally, the operations in process1400 may be used in conjunction with the systems and configuresdescribed elsewhere in the document. For example, at operation 1450, theunderlying mood models described with respect to FIGS. 2A-2D, 12, and 13may be used to determine a mood for the user and select contentresponsive to the mood. Moreover, the operations may be performed on theplayer 130 and/or the mood-based playlisting system 300 described inFIGS. 1 and 3, respectively. For convenience, particular components andmessaging formats described earlier are referenced as performing theprocess. However, similar methodologies may be applied in otherimplementations where different components are used to define thestructure of the system, or where the functionality is distributeddifferently among the components shown.

Initially, a user is enrolled in a training regimen (1410). For example,a user may be presented with a content selection and asked to specify aresultant mood. In another example, a user is asked to present a faceassociated with different moods. One example of a training regimen isshown in FIGS. 8-10.

Once the mood-based playlisting system has been trained, a camera isused to capture an image of the user (1420) and the image is analyzed(1430). For example, the image may be analyzed using the rules andconfigures described in FIGS. 12 and 13. In addition, the predicted moodmay be optionally determined using non-camera inputs (1440). Forexample, the mood-based playlisting system may determine a mood based ona selected station or channel or by reference a past historicalreference for a user's mood.

The mood is determined for the user (1450), and content responsive tothe mood is selected (1460). A camera is used to capture an image of theuser during or while perceiving the content (1470), and the image isanalyzed to determine an actual mood state (1480).

The mood-based playlisting system determines whether a predicted moodvaries from the actual mood (1480). If so, the variation is recorded andused when selecting content in the future (1485). If not, the mood-basedplaylisting system may record that the predicted mood was an accuratepredictor of the actual mood (1490). For example, the variation oraccuracy may be provided to the mood indicator library 340 described inFIG. 3.

FIG. 15 is an exemplary GUI 1500 that illustrates how different inputsmay be used as to a mood sensor into a mood-based playlisting system.While data provided by the sensors may differ from other data providedby other sensors in the mood-based playlisting system, the data providedby the sensors shown in GUI 1500 may be associated with different moodstates and used in a mood-based playlisting system. Furthermore, whilesome sensors may use a different form factor that constrains where thesensors may be used, using different sensors in different circumstancesenables the mood-based playlisting system to realize feature setsdifficult to otherwise achieve. For example, interface 1510 illustratesneural activity in imagery provided by a brain scan. In someimplementations, the form factor of the radiology equipment may limitthe use of the radiology equipment to laboratory and medicalenvironments. However, the radiology equipment may provide a degree ofaccuracy not easily obtained through other sensors. And, the dataprovided by the radiology equipment may be used to generate a moresophisticated model, which in turn may lead to more accurate results. Asshown, the interface 1510 indicates neural activity in a particular areaof the brain associated with a particular mood.

Interface 1520 illustrates data provided by an audio spectrum analyzer.Mood information may be derived by associating a certain frequency orarrangement of frequencies with a mood.

Interface 1530 illustrates data provided by an electronic signalmonitor. For example, synaptic activity indicative of a mood state maybe detected by a probe attached to the user. The relative intensity orfrequency may be used to indicate a mood state for the attached user. Inone implementation, a degree of tension may be determined and used togenerate the mood state.

Other implementations are within the scope of the following claims. Forexample, although the mood-based playlisting system has been describedin the context of a distributed system that may support multipledevices, the mood-based playlisting system may be distributed acrossmultiple systems and/or reside at a client device. One example of themood-based playlisting system being distributed across multiple devicesmay include having a portion of the mood-based playlisting system thatoperates in a data center where the content library and mood indicatorlibrary reside. The data center systems may interface with software thatoperates a mood calculator and content retrieval program to retrieve thecontent library from the central systems.

Alternatively, the mood-based playlisting system may be moreclient-focused and may perform more operations on the client. Forexample, the mood-based playlisting system described in FIG. 3 may beimplemented on a personal audio system. The personal audio system maystore multiple selections of content in memory and generate the playlistthat maintains the mood of the content that has been stored.Alternatively, the mood-based playlisting system may include anetwork-based device that implements the content selection andplaylisting on the client device but retrieves content from anetwork-based content library.

The mood-based playlisting system may be configured to preserve somemeasure of variation within the playlist. Thus, the mood-basedplaylisting system may be configured to recognize that if three countryballads having moods that have been gauged as depressing are played, theplaylist should then select a country song having a mood that has beengauged as uplifting. The variation rules may be described digitally anddistributed as programming criteria alongside or in conjunction withother licensing restrictions. For example, a license may govern thefrequency with which an artist or song may be played. In addition to thefrequency licensing restrictions, the content distributor may distributea mood-based playlisting rule set along with an electronic library toregulate access to the content.

Although the mood has been described in the context of content beingplayed, other techniques may be used to infer the mood. For example, theclient device may monitor how the user interfaces with the media player.Monitoring a volume level, monitoring changes to the volume level, andmonitoring whether a user changes an Internet radio station are examplesof operations that may be used to infer the mood. For example, when amedia player detects that a user reduces the volume level when a newtrack begins, the media player may determine that the user isexperiencing a less intense mood. In contrast, when the user increasesthe volume, the media player may determine that the user's moodintensifies.

The user interaction with the media player also may be analyzed withrespect to the content that is accessed. For example, if the user skipsto the next track immediately after accessing a new track, the mediaplayer may determine that the user's mood does not like the skippedtrack. The user's action may be extrapolated so that a mood that is theinverse of the mood of the rejected content is inferred. To illustrate,a user may initially select a country music Internet Radio station. Thesequence of content transmitted to the user may include a country rocksong, followed by a country ballad, followed by a country rock song.When the user listens to the first country rock song, skips the countryballad, and listens to the second country rock song, the media player(or host) may determine that the user's mood reflects a preference forcountry rock.

Although the description of a mood indication made distinctions betweenthe style and genre, the mood indications also may be made with respectto other factors, including the artist, the tempo, the era in which thecontent originated, the album, and/or other categorization. For otherforms of media (e.g., video or data), the mood indications may includemoods related to the identity of the producer, director, actors, and/orcontent rating (child, teen, all-ages) in addition to the category ofthe programming.

Analyzing the user's interactions to determine the mood is not limitedto the user's interaction with a media player. A user's interaction withan Instant Messaging program, an electronic mail program, or an InternetWeb browser are examples of other user activities that may be used todetermine the mood. Thus, when a user is typing quickly and exchangingmessages with many other users, an intense mood may be inferred. Incontrast, when the user is determined to be reading web pages at aslower pace, a relaxed mood may be inferred. The content in the userinteraction also may be used in determining the mood. Thus, the contentappearing in a web page accessed by the user may be used to determinethe mood for the user.

Although many of the previously described examples link a certainactivity or type of content with a certain mood, the examples illustratejust one mood that can be associated with an activity. Other moods maybe associated with the activity or type of content. A selection ofcontent or a user activity also may be associated with multiple moods.An example of content with multiple moods may include a song with anuplifting melody and depressing lyrics. A mood-based playlisting systemmay use either or both moods in selecting future content. If themood-based playlisting system sought to place an advertisement/productwith the uplifting mood indication, the mood-based playlisting systemmay incorporate the uplifting mood in the transition. If the mood-basedplaylisting system did not have an intended mood destination in mind,the mood-based playlisting system may continue to select content withmultiple mood elements to allow for an easier transition to a widervariety of content. A larger mood volume may represent the multipleelements with greater dimensions across multiple axes.

Although the mood-based playlisting system has been described usingplaylists, the mood-based playlisting system need not assemble an actualplaylist of songs. Rather, the content selection may be made on aselection-by-selection basis. The list of songs selected in this mannermay form a playlist.

Although the mood-based playlisting system has been described in thecontext of determining the mood state for a user, the mood-basedplaylisting system may be used to determine a mood state and selectcontent for a group of users. This may include selecting content forlarge Internet audiences. For example, the individual mood states forindividual members of a large audience may be aggregated to determine acollective mood state for the large audience.

In one example, the determining collective mood state may includesampling individual members of the audience for their mood states andusing the sampled mood information to generate a collective mood state.In another example, an audience listening to one content source may beanalyzed as a collection of groups. The mood-based playlisting systemmay analyze each individual group to determine whether the mood state ofthe group is consistent with the content being selected. When the moodstate for one of the groups indicates that the mood state for the groupis not compatible with the mood state for a content selection, themood-based playlisting system may reconfigure the selection of content.In one example, the group experiencing the mood state incompatibilitymay be transitioned to a different stream/playlist to preserve the moodstate compatibility. In another example, the mood-based playlistingsystem may select different content designed to retain the groupexperiencing the mood state incompatibility. This may includedetermining that more users are likely to be retained from the groupexperiencing the mood state incompatibility than are lost from othergroups not experiencing the mood state incompatibility.

The mood-based playlisting system may disperse and group users. Usersmay be grouped to reduce costs, to take advantage of discounts forlarger audiences, and to allow a limited pool of content to serve alarger community. This may include transmitting the same advertisementor segment lead to multiple users. The mood-based playlisting systemalso may disperse users from a common group. For example, a group ofusers may be accessing a host to access a popular selection of content.The mood-based playlisting system then may personalize the content basedon the determined mood so that the users are retained at a higher rateusing the mood-based playlisting system.

The mood-based playlisting system may normalize a mood indication to adesignated location or region. The normalization may be doneirrespective of whether targeted content is designated for the user. Forexample, the mood-based playlisting system may determine that operatinga playlist in a certain mood spectrum or volume retains listeners at agreater rate. In another example, the mood indication for the user isoperated in a specified range so that the user may be more receptive tocommunications delivered through other channels. This may include, forexample, an advertisement on television, an electronic mail message, atelephone call, a Web-based advertisement, or an instant message. In yetanother example, an advertiser may want a certain mood to be associatedwith a product. For example, a marketing firm may want a ‘happy’ moodassociated with the firm's content.

When calculating a mood transition, the mood-based playlisting systemmay reexamine the actual mood state during the transition and determineif the actual mood state matches the intended mood state. For example,although the mood state of the content may indicate that a user shouldbe in a relaxed mood, the user's activities on their client may indicatethat the user's mood state is not mellow (e.g., the user is experiencingstress or anxiety). The mood-based playlisting system may dynamicallyrespond to the actual mood state. In one example, the mood-basedplaylisting system may select content associated with a different mooddestination that is more compatible with the user's actual mood state.Thus, instead of playing an advertisement associated with a mellow mood,the mood-based playlisting system may select an advertisement with amood that is compatible with the actual mood of the user.

The mood based-playlisting system may include a detailed records systemfor reporting and accounting. For example, the mood-based playlistingsystem may record the moods of the user, the mood transition betweentracks, and the percentage of users that are retained for thetransition. Other records may include advertising effectiveness based onthe mood, and user listening habits (e.g., duration, user preferences).The records may be refined in an automated manner to develop moodtrending information. The mood-based playlisting system may generateautomated reports for system administrators and advertisers to improvethe enjoyment, effectiveness, and/or success of the mood-basedplaylisting system. This may include a report indicating that adifferent programming sequence may result in an increased response rateto an advertisement.

The mood-based reporting system may transmit several different sequencesof content to determine the relative efficacy of the differentsequences. The mood-based reporting system then may present the resultsto a system administrator and enable the system administrator to controlfuture content selection using the reported relative efficacyinformation. The reporting system may present results using differentmood metrics. For example, a first report may be based on only the moodof the content while a second report may gauge the user interaction withthe media player. The reporting system then may analyze the differences,and interpret the variation. The interpretation of the variation thenmay be used by a system administrator to plan future programming.

In another implementation, a computer program may be configured tomanage content accessed by a user by using a mood-based playlistingsystem to plot a transition from a first track to a second track usingat least one intermediary track. A mood originating point may bedetermined for the first track, and an intermediary track configured torealize a specified mood transition may be selected. The intermediarytrack may be rendered and a camera may be used to capture an image ofthe user during the rendering of the content. The image may be analyzedto realize an actual mood state, and the actual mood state may be usedto determine if the specified mood transition has been realized. Whenthe specified mood transition has been realized, the transition mayproceed. When the specified mood transition has not been realized, theactual mood may be used to select a different transition track.

In still another variation, a computer program may be configured tomanage electronic content made available to users. A first contentselection may be rendered, and a camera may be used to capture an imageof the user during rendering. The image may be analyzed to determine anactual mood indicator for the user. A mood spectrum may be determined todescribe other mood indicators that are consistent with the actual mood.Finally, a next content selection may be selected having a second moodindicator that lies within the mood spectrum.

Organization and Display of Visual Interfaces

A graphical user interface (GUI) may present mood information aboutusers in a network. Although earlier parts of this document describeplaylisting for users based on a user's mood information, in the contextof presenting mood information about more than one user, the term“co-user” is adopted. The term “co-user’ indicates that multiple users(e.g., co-users) are each part of a network of co-users and may viewmood information about each other. For example, a co-user's screen nameappearing in an America Online (AOL) Instant Messenger's Buddy List mayindicate the co-user's mood, such as “happy,” “sad,” “silly,” or“angry.” In another example, a co-user's mood may be more subtly definedas a combination of moods or a particular point on a spectrum of a moodfrom “happy” to “sad”, from “tired” to “energetic,” or from “good” to“evil.” The term “user” is adopted to describe a person who “owns,” oroperates the co-user list in which other users (e.g., “co-users”) arelisted along with the co-user's mood.

FIG. 16 is an exemplary graphical user interface (GUI) 1600 displaying alist of co-users organized by mood. The GUI 1600 includes a co-user list1601 with mood elements 1610-1613, and co-user elements 1620-1624. Aco-user list is user-defined in that the user may determine whichco-users are to be part of the user's co-user list. For example, a usermay have one hundred co-workers, but may only add fifty of theco-workers to the user's co-user list. Additionally, a co-user list ispresence conveying. A user may determine whether a particular co-user isavailable for communication based on the location of the co-user's namewithin the co-user list. For example, a co-user listed as “offline,” orhaving a grayed out co-user element, is not currently available forcommunication, while a co-user having a bright co-user element, or is inan “online” group, is currently available for communication.

Furthermore, a co-user list enables nearly instantaneous communications.The indication of a co-user's presence as available or unavailable forcommunication takes place in real-time. For example, as soon as a userlogs in to the co-user network, co-users having the newly logged in useron their co-user list may automatically and instantaneously perceive theuser's new status.

A co-user list is also “noisy” in that the co-user list may overlap anyother ongoing application interfaces. For example, a co-user list may beperceivable on a co-user's computer desktop (or other device, such as acellular phone, personal digital assistant, etc.) regardless of whatother applications are currently running and whether the user isconstantly, or consistently, paying attention to the co-user list.

The four mood elements shown in FIG. 16 include a happy mood element1610, an angry mood element 1611, a sad mood element 1612, and a sillymood element 1613. Each mood element represents a mood that may beassociated with one or more co-users.

The GUI 1600 also includes co-user elements 1620-1624. The co-userelements 1620-1624 each represent a particular co-user. For example,Karan's co-user element 1620 represents the co-user who maintains Karanas their online identity. The string “Karan” may be a co-user ID, suchas an AOL Instant Messenger screen name, or alternatively the co-usermay have associated an equivalent network ID with an actual name ornickname.

A co-user may be associated with one of the moods related to the fourmood elements. Each of the co-user elements 1620-1624 may be positionedbelow a particular mood element enabling a viewer to perceive the moodsof co-users represented by co-user elements 1620-1624. Karan's co-userelement 1620 and Lester's co-user element 1621 are positioned below thehappy mood element 1610, indicating that Karan and Lester are each in ahappy mood. Jordan's co-user element 1622 and Louis's co-user element1623 are positioned below the angry mood element 1611 to indicate thatJordan and Louis are each in an angry mood. Brian's co-user element 1624is positioned below the sad mood element 1612 to indicate that Brian isin a sad mood. The “(2)” 1651 displayed to the right of the silly moodelement indicates that two of the co-users are in a silly mood. Therightward facing arrow 1652 indicates that the silly co-users are hiddenfrom display. A co-user may view the silly co-users by clicking on therightward facing arrow to update the co-user GUI 1600 to include co-userelements for the silly co-users.

In some implementations, a co-user's mood may be represented in a wayother than in reference to a correlating mood element, such as moodelements 1610-1613. Changes in a font used to display a co-user element,such as co-user elements 1620-1624, also may be used to indicate mood.For example, bright, large lettering may be used to indicate a happymood, while small, dingy lettering may be used to indicate a sad mood.Furthermore, a user may be able to right-click on a co-user element toreveal a mood panel. The mood panel may indicate or suggest a mood forthe co-user by way of, for example, a mood score (e.g., a mood score of95 relates to happy, while a mood score of 15 relates to sad), a moodlabel (e.g., “happy”), or an icon (e.g., a smiley face icon indicates ahappy mood). Each of the numerous methods described in this document forindicating a co-user's mood is individually sufficient to indicate themood of a particular co-user. However, use of more than one of themethods for indicating mood may be desired by a particular co-user andthus may be so provided.

In the implementation depicted in FIG. 16, each mood element representsa particular mood. In another implementation, a mood element may beconfigured to represent a mood that is a combination of two or moremoods. For example, instead of having a single mood element to representa happy mood, there may be two mood elements: a first mood elementserving to identify co-users who are happy and energetic, and a secondmood element serving to identify co-users who are happy and relaxed.

In one implementation, a co-user list has an organizational structurethat is distinct from an organization based on mood. For example, aco-user list may be organized based on the relationship of each listedco-user to the user who owns the co-user list. Alternatively, themood-based organization may be displayed in conjunction with thenon-mood-based organization. For example, a co-user list may beorganized based on relationships and the co-users within eachrelationship may be further organized based on mood.

FIG. 17 is an exemplary graphical user interface (GUI) 1700 for a listof co-users organized by mood. More particularly, FIG. 17 shows aco-user list 1701 where co-user elements are displayed hierarchically tocategorize users based on both a mood attribute and a non-moodattribute. The top level of the hierarchical display includes non-moodattribute elements 1710-1711. The second level of the hierarchicaldisplay includes mood elements 1720-1724, which are similar to moodelements 1610-1613 of FIG. 16. The lowest level of the hierarchicaldisplay includes co-user elements 1730-1735, which are similar toco-user elements 1620-1624 of FIG. 16.

The non-mood attribute elements 1710-1711, mood elements 1720-1724, andco-user elements 1730-1735 are arranged in a hierarchical fashionenabling a user to perceive both mood and non-mood attributes of theco-users within the co-user list. For example, because Karan's co-userelement 1730 is located below the happy mood element 1720, and the happymood element 1720 is located below the friends mood element 1710, a usermay easily perceive that Karan is a friend (the non-mood attribute), andthat Karan is in a happy mood (the mood attribute). Another usefulaspect of associating mood organization with non-mood attributeorganization is the ability of the user to monitor the moods of only asub-set of co-users, for example, where a user is only interested incommunicating with family members who are happy.

Co-user list 1701 also shows co-user elements categorized based on moodattributes. More particularly, the co-user list 1701 includes a toplevel category 1712 indicating the portion of co-user list 1701displaying co-user elements based on mood attribute. The second level ofthe display beneath category 1712 includes mood elements 1725-1728. Moodelements 1725-1728 correspond to mood elements 1720-1724 in that themood category of co-user list 1701 includes all moods listed beneath thenon-mood attributes 1710-1711. The lowest level of the display beneathcategory 1712 includes co-user elements 1730-1735, which are the sameco-user elements as displayed beneath non-mood attributes 1710-17111.

For example, Karan's co-user element 1730, along with co-user element1731 for Lester and co-user element 1733 for Marcus, are listed belowthe happy mood element 1725. A user may perceive that Karan, Lester andMarcus are in a happy mood due to their placement under happy moodelements 1720 and 1722 in the non-mood attribute display portion ofco-user list 1701. However, if a user wishes to perceive all co-usersthat are in a happy mood, top level category 1712 organizes co-userelements within the happy mood element 1725 in the mood-based attributeportion of co-user list 1701. Thus, the user need not scan each of thenon-mood attribute element portions of co-user list 1701 seekingco-users in a happy mood.

To group co-user elements by mood within a non-mood group, such asfriends, a user may right-click on the non-mood group label. By doingso, the user may be presented with a menu where the user may select anitem labeled, for example, “organize co-users within this group bymood.” The co-users within the friends group may then be furtherorganized based on mood information associated with each of the friendco-users. Alternatively, if a user no longer wishes to view the co-userlist organized based on the mood attributes, the user may againright-click on the friends label and select an item labeled, forexample, “do not organize co-users within this group by mood” from themenu. Thus, the display of the co-user list is responsive to the user'srequest and is based on the user's preferences. Furthermore, the co-userlist may be displayed differently in response to a change in the user'spreference.

In one implementation, co-users are grouped into moods that arepredefined by an intermediary, such as a co-user list. A co-user may beassigned to a particular group by determining which group is associatedwith a mood that most closely resembles a mood of the co-user. Inimplementations where moods are defined by a multi-dimensional space, aparticular group may be selected, or organized, by finding the minimumdistance between the determined mood value for the co-user and areference mood value for each of more than one particular mood states.

In another implementation, a mood element may be configured to representa combination of mood and non-mood information. For example, a moodelement may indicate (1) that a particular co-user is sad, and (2) thelikelihood that the sad co-user may correspond with other co-users whilethe co-user is in the sad mood, where the likelihood may be representedby, for example, the number of times in the past the sad co-user hascorresponded with other co-users while in a sad mood or the percentageof times the co-user has corresponded with other co-users while in a sadmood.

In yet another implementation, some or all of the mood groups may bedefined by the user. For example, a user may define a mood group bygiving the mood group a name and indicating portions of themulti-dimensional space for which the mood group is representative.

FIG. 18 is a visual representation of mood values in a three dimensionalspace 1800. More particularly, FIG. 18 shows a three dimensional space(i.e., a mood space) 1800 with three axes 1801-1803, such that each axiscorresponds to a particular aspect of a mood. The happiness dimension1801 ranges from 0 to 10 and is associated with the happiness of aco-user, with 0 being the least happy value, and 10 being the most happyvalue. The energy dimension 1802 ranges from 0 to 10 and is associatedwith the energy exhibited by a co-user, with 0 being the leastenergetic, and 10 being the most energetic. The “busyness” dimension1803 ranges from 0 to 10 and is associated with an amount of work that aco-user is doing, with 0 being idle, and 10 being extremely busy. Anexemplary mood value is illustrated by point 1805, having a value of (5,5, 5). Point 1805 represents a co-user mood having medium values for allthree mood aspects.

In another implementation, a mood space may have dimensions that aredifferent from the exemplary mood space shown in FIG. 18. For example,other mood dimensions may include physical wellness, excitement, orhunger. Furthermore, the range of values for dimensions may differ fromthe example of FIG. 18. For example, the happiness dimension may rangefrom −10 to +10, with −10 being the least happy value (e.g., miserable),0 being neutral (e.g., content), and +10 being the most happy value(e.g., ecstatic). In yet another implementation, the mood valueassociated with a co-user may be represented as string of textdescribing the mood as calculated by a mood engine. In still anotherimplementation, a mood may be represented as a vector of one or morevalues, where each value in the vector represents a measurement of aparticular aspect of a co-user mood.

The mood value of a co-user may be displayed in conjunction with aco-user element. For example, the mood value of (5, 5, 5) may bedisplayed next to a co-user element for a co-user having this particularmood value. In another example, the mood value (5, 5, 5) may bedisplayed in a mood panel that appears when a user mouses over, orright-clicks, the co-user element for the co-user having this moodvalue.

FIG. 19 is an exemplary data structure 1900 for storing the mood valuesof co-users. Mood information may be stored in a relational databasesystem that logically organizes data into a database table. The databasetable arranges data associated with co-user mood values in a series ofcolumns 1910-1913 and rows 1920-1926. Each column 1910-1913 representsan attribute of a co-user's mood value and each row 1920-1926 representsa collection of attribute values for a particular co-user's mood value.The attributes are used to associate co-users with moods, to displayusers organized by mood, as seen in the co-user UIs of FIGS. 16 and 17,and to sort co-users by mood value.

The attribute 1910 includes a co-user ID to identify a co-user. Theco-user ID corresponds to a particular co-user in a co-user list. Theco-user mood value data structure 1900 also includes a Happy Value 1911,an Energy Value 1912, and a Busy Value 1913. Each value represents onedimension in a mood space, such as the mood space of FIG. 18. The HappyValue corresponds to the Happiness Axis 1801, the Energy Value 1912corresponds to the Energy Axis 1802, and the Busy Value 1913 correspondsto the Busy Axis 1803.

FIG. 20 is an exemplary data structure 2000 configured to define moodsin terms of mood values. The mood-defining values 2000 may be stored ina relational database system that logically organizes data into adatabase table. The database table arranges data associated with mooddefining values in a series of columns 2010-2013 and rows 2020-2031.Each column 2010-2013 represents an attribute of a mood-defining valueand each row 2020-2031 represents a collection of attribute values for aparticular mood definition. The attributes may be used to define aparticular mood, such as happy or angry, by defining a portion of themood space corresponding to the particular mood.

The attribute 2010 includes a mood type to identify a mood. The mooddefining data structure 2000 also includes a Happy Value 2011, an EnergyValue 2012, and a Busy Value 2013. Each value represents the value ofone dimension of a point in a mood space, such as the mood space of FIG.18. As described above with respect to FIG. 19, the Happy Value 2011corresponds to the Happiness Axis 1801, the Energy Value 2012corresponds to the Energy Axis 1802, and the Busy Value 2013 correspondsto the Busy Axis 1803. FIG. 21 is an exemplary data structure forstoring a list of co-users. The list of co-users may be configured by auser, and stored electronically at the user's location. For example, auser of AOL's Instant Messenger service (AIM) may use AIM to exchangeinformal messages with friends. The co-user list, and associated values2100, may be stored in a relational database system that logicallyorganizes data into a database table. The database table arranges dataassociated with co-users in a series of columns 2110-2112. Each column2110-2112 represents an attribute of a co-user and each row 2120-2126represents a collection of attribute values for a particular mooddefinition. The attributes may be used to describe characteristics ofco-users displayed in a co-user list, such as GUI 1600 of FIG. 16.

Each value column 2110 stores a screen name attribute for a particularco-user. The screen name of a co-user is a unique identifier for thatco-user. As shown, each row of column 2111 stores a full name attributefor a particular co-user. The full name is the actual name of a co-user.Each row of column 2112 stores a group attribute for a particularco-user. The group attribute places each co-user into a particular groupof co-users. For example, a visual element representing a group(non-mood attribute element 1710) may be displayed, as describedpreviously with respect to FIG. 17.

FIG. 22 is a flow chart 2200 showing an exemplary process by which aco-user may be associated with a particular mood. Generally, theoperations shown in flow chart 2200 may be used in conjunction with thesystems and configurations described earlier in, for example, FIG. 1.More particularly, the operations shown in flow chart 2200 may beperformed on content source 110 of FIG. 1. Similarly, the systems usedin flow chart 2200 may use the GUIs described with respect to, forexample, FIGS. 16, 17, 23, 25-28. For convenience, particular componentsand messaging formats described earlier are referenced as performing theprocess. However, similar methodologies may be applied in otherimplementations where different components are used to define thestructure of the system, or where the functionality is distributeddifferently among the components shown.

Initially, content source 110 accesses a list of co-users (2210). Thelist of co-users may be electronically stored at a user location where auser at the user location has preconfigured a list of co-users. Forexample, as was described with respect to FIG. 21, a list of co-usersmay be stored in a database at a user location.

Content source 110 then selects a first co-user from the list ofco-users (2220). For example, the host may access the first row 2120 ofthe co-user list 2100, as described with respect to FIG. 21, to select afirst co-user.

Content source 110 then accesses the mood value of the first co-user(2230). For example, content source 110 may access the values in the rowof co-user mood value database 1900 where the screen name is the same asthe screen name of the first co-user selected in 2220. In anotherimplementation, the mood value for a particular co-user may be stored atother locations, such as at the co-user's location or a servercontrolling network communication between the first co-user and otherco-users.

Content source 110 then accesses mood definitions to determine the firstco-user's mood (2240). The first co-user's mood is determined based onthe mood value accessed in operation 2230 from a relational database,for example, the relational database of FIG. 20. As shown in FIG. 20,the mood definition database 2000 associates moods with a particularmood value. However, a single mood may be associated with more than onemood value. Content source 110 determines the first co-user's mood byretrieving the first co-user's mood value, as accessed in operation2230, and determining the corresponding mood from the mood definitiondatabase 2000.

Next, content source 110 associates the first co-user, as selected inoperation 2220, with the mood determined in operation 2240 (2250). Toassociate a co-user with a mood from the mood definition database,content source 110 may access the mood values in each row of the mooddefinition database and calculate the vector distance from the co-user'smood value to the reference mood values accessed from the database. Whenvectors are used, multiple mood calculations are independently made. Themood definition having a reference mood value that has the smallestdistance from the mood value of a co-user may be designated as the moodof that co-user.

Content source 110 next determines whether there are more co-users forwhom moods should be determined (2260). If so (2264), the host proceedsby analyzing the mood of a second co-user (2220). If the host determinesthat there are no more co-users whose moods need to be determined(2268), then the association operations described with respect toprocess 2200 end (2270).

Once a mood has been associated with one or more co-users, a co-userlist, such as GUIs 1600 and 1700 of FIGS. 16 and 17, may be configuredto organize co-users by mood, for example. In addition to presentingmood information via the organization of the co-user list, a co-userelement included in a co-user list may be displayed in a way thatenables to the mood of each co-user in the co-user list to be perceivedby the user.

FIG. 23 is an exemplary graphical user interface (GUI) 2300 featuringco-user elements in a manner that is representative of a mood. GUI 2300includes a list of co-user elements 2310-2315. Each co-user element isrendered according to the corresponding mood of the co-user. Forexample, co-user element 2310 represents the co-user with the screenname Karan. The co-user element includes an icon 2321, a screen name2322, and a mood descriptor 2323.

FIG. 24 is an exemplary data structure 2400 of a configurationindicating how a mood may be expressed within a co-user element. Datastructure 2400 includes visualization data associated with differentmoods. The visualization data is arranged within data structure 2400 ina series of columns 2410-2415 and rows 2420-2424. Each column 2410-2415represents an attribute of a co-user element GUI, and each row 2420-2424represents a collection of co-user element GUI values for a particularmood.

Column 2410 includes a list of mood names that identify various moods.The mood name may be used to provide mood information for a co-userusing, for example, the mood classifications process described earlier(e.g., process 2200 of FIG. 22).

The visualization data in data structure 2400 also includes column 2411for icons associated with each mood name; column 2412 for a font nameassociated with each mood name; column 2413 for a font size associatedwith each mood name; column 2414 for a font color associated with eachmood name; and column 2415 for a sound associated with each mood name.An icon may include an image that is representative of the mooddescribed by the mood name in column 2410. An example of an icon mayinclude a cartoon face with a smile 2410 a to indicate a happy mood. Thefont name 2412 indicates the font to be used for displaying the textportion of the co-user element. The font size 2413 indicates the size offont to be used for displaying the text portion of the co-user element.The font color 2414 indicates the color of the text included in theco-user element. The sound 2415 indicates a sound that may be playedwhen a user interacts with a co-user element through the co-user list.Such an interaction may include mousing-over, clicking upon, ordouble-clicking upon the co-user element.

FIG. 25 is an exemplary graphical user interface (GUI) 2500 with co-userelements sorted based on a mood-based value. As shown in GUI 2500,co-user elements are sorted alphabetically by the mood associated witheach co-user element. GUI 2500 includes co-user elements 2510-2516. Eachof co-user elements 2510-2516 includes an icon, for example, a sad,crying face 2546, a co-user screen name, for example, “Nida” 2536, and aco-user descriptor, for example, “Very Sad” 2526. Mood descriptors2520-2526 represent a mood-based value, for example, a mood indicator.

The first three co-user elements 2510-2512 displayed represent co-usersMelvin, Jennifer and Mary respectively. Melvin, Jennifer and Mary are ina happy mood, represented by co-user descriptors 2520-2522. In oneimplementation, co-user elements may be sorted based on the distancebetween a co-user's mood value and a reference mood value. Where moodvalues are represented as multidimensional vectors, as in the co-usermood value database 1900 of FIG. 19, the difference between the moodvalue of a co-user and a reference mood value may be the distancecalculated between the two values using a standard vector distancecalculation. The co-user elements may be sorted from greatest to leastdistance, or from least to greatest distance. Where a mood value isrepresented as a vector (e.g., as shown in FIG. 18), Melvin may have amood value that is more similar to the reference mood value associatedwith a happy mood than either of Jennifer's or Mary's mood value. Hence,a co-user element associated with Melvin may be placed at a specialposition in the co-user list to indicate that Melvin is more happy thaneither of Jennifer or Mary. In another implementation, the mood valuemay be represented as a string associated with the mood, in which casethe order in which co-user elements 2510-2512 are displayed is notindicative of a relative distance between the mood values of Melvin,Jennifer and Mary, and the reference mood value.

GUI 2500 includes a menu 2502 for selecting the sort order of theco-user elements. The menu selection 2503 is set to sort the moods by“Mood Compatibility.” In one implementation, sorting by moodcompatibility includes sorting where the mood value is the mood value ofthe co-user. GUI 2500 includes a co-user mood value display element2504, which indicates that the co-user is in a happy mood.

In yet another implementation, co-user elements may be sorted by a valuethat indicates the difference between the mood value of the co-user, anda reference mood value. For example, several co-users may be determinedto be in a happy mood because the mood value associated with eachco-user is closest in value to a reference value for a happy mood.However, although each of the several co-users have a mood value that isclose to the reference value for a happy mood, each of the severalco-users may not necessarily have the same mood value. In this case,some co-users may be happier than others in that the mood valueassociated with some co-users may be mathematically closer to thereference mood value for a happy mood value. Hence, the co-user elementsrelated to the several co-users having a happy mood may be sorted to notonly indicate the happy mood, but also to indicate the relativehappiness of each co-user based on the distance of each co-user'sassociated mood value from the happy mood reference mood value.

FIG. 26 is an exemplary graphical user interface (GUI) 2600 enabling auser to configure how mood information related to the user may be sharedwith the co-users in the user's co-user list. Radio buttons 2610-2613enable a user to select one of four possible modes to share access tomood information. As shown, only one radio button may be selected at atime (e.g., selecting one radio button will un-select any previouslyselected radio button). Due to privacy concerns, a user may not wish toenable access to the user's mood information by co-users. For example, auser may selectively enable access to the user's mood information bysome co-users, but not others, for example, by selecting radio buttons2620 or 2640. In addition, a user may wish to limit the type of moodinformation shared with co-users, for example, on a per co-user or groupbasis, by selecting, for example, radio button 2640.

If the first radio button 2610 is selected, a user's mood value may beaccessed by all co-users on the same network. If the second radio button2611 is selected, a user's mood value may be accessed by “buddies only,”in other words, only by co-users that were selected as “buddies” by theuser. If the third radio button 2612 is selected, a user's mood valuemay not be shared with any other network co-users.

Additionally, although not shown in FIG. 26, a user may restrict accessto the user's mood value based on a class of co-users. A class may bedefined as a group of co-users based on age, demographics, profileinformation, including likes and dislikes, membership of anorganization, subscription to an online service provider, or the like.Furthermore, a user may restrict access to the user's mood value basedon a group into which the co-users fall. For example, a user may wish todisplay the user's mood value to the co-users located in a family groupin the co-user list, but not co-users organized into a co-workers group.

The fourth radio button 2613 enables a user to grant permission to viewmood information on a per co-user basis. When the fourth radio button2613 is selected, the list view 2614 is activated. A user may select oneor more co-users from the list view. The list view 2614 shows thatco-user Jordan is selected 2615. The checkbox 2617 displayed below thelist view 2614 enables a user to specify if the selected co-user is ableto access the mood information belonging to the user, and is empty ifthe selected co-user does not have permission to access the moodinformation. The user may change the accessibility of mood informationby the co-user by clicking on the checkbox 2617 insert, or remove, acheckmark.

In one implementation, a user may specify a range of mood values thatthe user elects to make accessible to a particular co-user, or group ofco-users, or block with respect to such a particular co-user or group ofco-users. For example, a user may not wish to share moods, such ashappy, sad, or angry, with work colleagues because the user has deemedsuch moods as inappropriate for the work place. At the same time, a usermay be willing to share with co-workers a mood which describes the“busyness” level of the user.

In one implementation, a user may be presented with a set of predefinedmoods from which the user may specify a subset of moods that may beshared. The specified subset of “presentable” moods may be referred toas the projectable mood of the user. Because the projectable moods areuser specific, the projectable moods may vary between users.

In other implementations, a user may elect to completely block allco-users from viewing the user's mood. In this configuration, no moodelement (e.g., an icon, a text style, etc. as described previously) maybe displayed along with the co-user element of a -user who has noelected. In one configuration, when a user searches for co-users havinga particular mood or moods, a co-user element, for whom mood informationis not available, may not be retrieved as a result of such a search whena blocking configuration is in effect. Alternatively, the co-userelement may be retrieved as a result of a search for co-users having aparticular mood or moods without any indication of “blocking” aco-user's actual mood. The difference in whether a result is returnedmay be a default determination or may optionally be designated by theuser who has elected to block all mood information.

In addition to seeking co-users within a user's co-user list having aparticular mood or moods, a user may search for co-users in the co-usernetwork that are not in the user's co-user list having a particular moodor moods. The user may then add the newly found co-users to the user'sco-user list. The search for new co-users to add to the user's co-userlist may be based at least in part on mood information. For example, auser may wish to communicate with a co-user in a happy mood, but none ofthe co-users that are in the user's co-user list and online arecurrently in a happy mood. Therefore, the user may search for newco-users having a happy mood with whom the user may engage in a singlecommunication session or add to the user's co-user list. In anotherexample, a user may search for new co-users to add to the user's co-userlist that are presently in a happy mood or previously identified asbeing in a happy mood.

FIG. 27 is an exemplary graphical user interface (GUI) 2700 configuredto enable a user to perceive a mood determined for the user (via acalculated mood value) by the system and to provide feedback as to theaccuracy of the calculated mood value, thereby correcting or eventraining the mood determination system. GUI 2700 includes co-userelements 2730-33. GUI 2700 also includes a co-user mood indicator 2705,which indicates that a mood engine has determined that the user is in an“upset” mood 2706. A pop-up menu 2710 may be visible in response toaright click by the user on the co-user mood indicator 2705. The usermay specify an actual mood in the pop-up menu 2710 that is differentfrom the mood indicated by the co-user mood indicator 2705. Byspecifying an actual mood that is different from the determined mood,the user may provide negative feedback. For example, the “busy” mood2715 may be selected. Alternatively, if a user does not specify anactual mood that is different from the determined mood, the user isimplicitly providing positive feedback indicating that the system hascorrectly identified the user's mood. Enabling a user to specify anactual mood (e.g., provide negative feedback), or inferring a correctmood when the user fails to specify an actual mood (e.g., providepositive feedback) may be used to train the mood recognition engine andimprove its accuracy in detecting a user's mood.

FIG. 28 is an exemplary graphical user interface (GUI) 2800 configuredto enable a user to specify animations that may be presented with one ormore moods for a co-user. Animation may be used to represent a co-user'smood. The animation may be associated with a particular co-user. Forexample, while a stationary smiley face is typically presented in theabsence of consideration of mood, a “bouncing smiley face” may bepresented when a co-user is happy. Such an animation may only beprovided to co-users in the user's friends group if the user hasdetermined, and specifies, that the “bouncing smiley face” animation isnot appropriate for co-users in the user's co-workers group. Theanimation may appear in a co-user list, in a separate GUI elementcorresponding to a particular co-user, or in an instant messagingconversation window. For example, GUI 2800 includes a co-user animation2810 and a co-user animation 2811, both of which are displayed in aninstant messaging conversation window. The co-user animation 2810 isanimated in a way that reflects the mood of a user participating in theconversation and co-user animation 2811 reflects the mood of theco-user, with whom the user is communicating.

The mood-based animation associated with a particular co-user mayinclude an animated character that performs actions based on the mood ofthe co-user. In another example, an animation may include multiplecharacters that are each associated with a particular co-user. Themultiple characters may interact with one another based upon the mood ofthe co-users that the characters represent and/or upon attributes of thecommunication between the co-users.

Although this description has focused on providing mood elements forco-users to indicate a mood of co-users within a co-user list, the useof mood-identifying elements need not be so limited. Mood elements alsomay be employed in other contexts in order to display a mood for a user.For example, mood elements may be displayed alongside a user element(such as a screen name) in an email message. In another example, a moodelement may be provided alongside a user element in an instant messagingwindow or a chat room window. In yet another example, mood elements maybe displayed in association with a user element when the user elementindicates that a particular user has visited a webpage, reviewed anarticle, or purchased an item. In these examples, a mood element for auser may be displayed in association with a user element, but without anindication of an online status of the user.

In some implementations, a mood element for a co-user may be displayedin a co-user list even if an online status of the co-user isunavailable. For example, a co-user may set a privacy setting thatprevents users from viewing whether the co-user is online or offline. Aco-user list may be configured to display the co-user element for theco-user with an indication that the online status of the co-user isunknown. However, a current mood, or a previous mood, may still bedisplayed as a mood element associated with the co-user element.

In some implementations, a mood may be associated with a co-user andprovided to a user indirectly, rather than being provided as a moodelement displayed with a co-user element. For example, a user may wishto send an e-mail message to co-users listed in an address bookassociated with the user who are in a particular mood, such as a happymood. The e-mail message may include an e-mail message generated by theuser or an e-mail message, such as an electronic party invitation,generated by a third party web site or service that has access to theuser's address book for contacts. In this configuration, a user maycommunicate with a co-user based on a mood associated with the co-userwithout being provided with, and viewing, the co-user's mood. Otherexamples include sending a short message service (SMS) text message to aco-user via a cell phone or PDA, based on a mood associated with theco-user. For example, a user may select a short message service (SMS)message “recipient” as “all contacts who are happy.” Again, the user maycommunicate with a co-user based on a mood of the co-user without havingthe co-user's mood displayed to the user.

The described systems, methods, and techniques may be implemented indigital electronic circuitry, computer hardware, firmware, software, orin combinations of these elements. Apparatus embodying these techniquesmay include appropriate input and output devices, a computer processor,and a computer program product tangibly embodied in a machine-readablestorage device for execution by a programmable processor. A processembodying these techniques may be performed by a programmable processorexecuting a program of instructions to perform desired functions byoperating on input data and generating appropriate output. Thetechniques may be implemented in one or more computer programs that areexecutable on a programmable system including at least one programmableprocessor coupled to receive data and instructions from, and to transmitdata and instructions to, a data storage system, at least one inputdevice, and at least one output device. Each computer program may beimplemented in a high-level procedural or object-oriented programminglanguage, or in assembly or machine language if desired; and in anycase, the language may be a compiled or interpreted language. Suitableprocessors include, by way of example, both general and special purposemicroprocessors. Generally, a processor will receive instructions anddata from a read-only memory and/or a random access memory. Storagedevices suitable for tangibly embodying computer program instructionsand data include all forms of non-volatile memory, including by way ofexample semiconductor memory devices, such as Erasable ProgrammableRead-Only Memory (EPROM), Electrically Erasable Programmable Read-OnlyMemory (EEPROM), and flash memory devices; magnetic disks such asinternal hard disks and removable disks; magneto-optical disks; andCompact Disc Read-Only Memory (CD-ROM). Any of the foregoing may besupplemented by, or incorporated in, specially-designed ASICs(application-specific integrated circuits).

It will be understood that various modifications may be made withoutdeparting from the spirit and scope of the claims. For example, usefulresults still could be achieved if steps of the disclosed techniqueswere performed in a different order and/or if components in thedisclosed systems were combined in a different manner and/or replaced orsupplemented by other components. As another example, a co-user elementis used throughout to represent a unique identifier of an account, butany other unique identifier of an account may be used when linkingaccounts. Accordingly, other implementations are within the scope of thefollowing claims.

1.-24. (canceled)
 25. A method for rendering information on a display,the method comprising: rendering one or more co-user elements, eachco-user element including an identity element, the identity elementproviding a basis to enable a user to perceive a co-user identity;rendering one or more mood elements structured and arranged to enable auser to perceive a mood associated with the co-user identity;calculating a mood value associated with the co-user identity, the moodvalue providing a basis to arrange the one or more co-user elements; andarranging more than one co-user elements in a hierarchical manner basedon at least one of the mood value and a non-mood attribute.
 26. Themethod of claim 25, wherein calculating the mood value comprisescalculating the mood value using metadata associated with contentcurrently accessed by the user.
 27. The method of claim 25, furthercomprising: comparing the mood value to a list of moods; and determiningthe mood associated with the co-user identity based on the mood value.28. The method of claim 25, further comprising: representing the moodvalue associated with the co-user identity as a multi-dimensionalrating, whereby each rating is associated with a particular aspect ofthe mood associated with the mood value.
 29. The method of claim 25,further comprising: representing the mood value by changes in a fontused to display the one or more co-user elements.
 30. The method ofclaim 25, wherein the one or more mood elements is configured torepresent a combination of two or more moods.
 31. The method of claim25, wherein the one or more mood elements is configured to represent acombination of mood and non-mood information.
 32. The method of claim25, further comprising: displaying the mood value in a three dimensionalspace.
 33. The method of claim 32, wherein the three dimensional spacecomprises three axes, each axis corresponding to a particular aspect ofthe mood associated with the mood value.
 34. The method of claim 25,wherein the non-mood attribute comprises a relationship associated withthe user and one or more co-users.
 35. A tangible computer-readablestorage medium encoded with a computer program instructions that whenexecuted by a processor, cause the processor to perform a method forrendering information on a display, the method comprising: rendering oneor more co-user elements, each co-user element including an identityelement, the identity element providing a basis to enable a user toperceive a co-user identity; rendering one or more mood elementsstructured and arranged to enable a user to perceive a mood associatedwith the co-user identity; calculating a mood value associated with theco-user identity, the mood value providing a basis to arrange the one ormore co-user elements; and arranging more than one co-user elements in ahierarchical manner based on at least one of the mood value and anon-mood attribute.
 36. The tangible computer-readable storage medium ofclaim 35, wherein the mood value is calculated using metadata associatedwith content currently accessed by the user.
 37. The tangiblecomputer-readable storage medium of claim 35, further comprisingcomputer program instructions executed by the processor for: comparingthe mood value to a list of moods; and determining the mood associatedwith the co-user identity based on the mood value.
 38. The tangiblecomputer-readable storage medium of claim 35, further comprisingcomputer program instructions executed by the processor for:representing the mood value associated with the co-user identity as amulti-dimensional rating, whereby each rating is associated with aparticular aspect of the mood associated with the mood value.
 39. Anapparatus for displaying information about co-users, the apparatuscomprising: a processor configured to render one or more co-userelements, each co-user element including an identity element, theidentity element providing a basis to enable a user to perceive aco-user identity; and a display coupled to the processor and configuredto render one or more mood elements structured and arranged to enable auser to perceive a mood associated with the co-user identity, and rendera user feedback element structured and arranged to receive user feedbackregarding accuracy of the mood determined and associated with theco-user identity.
 40. The apparatus of claim 39, wherein the processoris further configured to calculate a mood value associated with aperceived mood using metadata associated with content currently accessedby the user.
 41. The apparatus of claim 39, wherein the one or more moodelements is configured to represent a combination of two or more moods.42. The apparatus of claim 39, wherein the one or more mood elements isconfigured to represent a combination of mood and non-mood information.